Skip to main content

Lessons from a comparison of immuno-chromatographic and chemiluminescent micro-particle immunoassay in the diagnosis of syphilis

Abstract

Objective

To synthesize lessons from comparison of results obtained from the immuno-chromatographic SD Bioline testing method and the chemiluminescent micro-particle immunoassay Architect in the diagnosis of syphilis at Livingstone Central hospital laboratory.

Results

The specificity and sensitivity of SD Bioline syphilis 3.0 against the chemiluminescent immunoassay using the Architect syphilis Treponema pallidum (TP) was 85.3% and 91.3% respectively with substantial agreement between the two test methods (88%, ĸ  = 0.76; p < 0.0005). We recommend further comprehensive study with a larger sample size and clinical details to ascertain the validity of our findings. We also recommend using a non-treponemal test with the current treponemal tests being used to aid diagnosis.

Introduction

The laboratory serologic diagnosis of syphilis can be complicated [1] especially with limited diagnostic tools owing to the natural course of the infection which is characterized by periods with and without clinical manifestations [1]. Furthermore, the disease stages show variable sensitivity and specificity for some test methods in detecting Treponemal pallidum (TP) antibodies [1, 2]. It is therefore mandatory to interpret laboratory results on the basis of patient history and physical examination findings [3]. The traditional screening of syphilis recommended by the Center for Disease Control and Prevention (CDC) is the use of a nontreponemal test, such as rapid plasma reagin (RPR) [4]. However, the use of chromatographic and chemiluminescence immunoassay (CIA) treponemal test methods for screening syphilis is empirical and gaining acceptability [5]. It is important and mandatory to evaluate the diagnostic performance of laboratory test methods in every laboratory so as to instill confidence in the clinician and patient regarding the reliability of laboratory results [1]. Several reports have imaged that have compared the SD Bioline syphilis test with other treponemal methods, stressing the importance of conducting pre-clinical and -laboratory performance and applicability of all testing methods for syphilis diagnosis [6, 7]. At Livingstone Central hospital, which is based in the Southern Province of Zambia, we use the SD Bioline syphilis test only (recently introduced), as an index test to aid in the diagnosis of syphilis for patients while Architect syphilis TP is specifically used for screening blood donors. The disease burden of syphilis in our region is unknown owing to paucity of studies. We decided to perform an exploratory study whose objective was to compare the SD Bioline syphilis test performance against the Architect syphilis TP. This was prodded by the clinicians’ concern for the high TP positivity rate recorded for the months, March to May, 2017 using the SD Bioline syphilis test.

Main text

Methods

Study design

This was a cross-sectional study conducted at the Livingstone Central hospital laboratory research section.

Setting and sample size

Data collection was planned after the index test but before the reference standard was performed. We collected data for the SD Bioline syphilis 3.0 positivity rate for the months of March, April and May, from the laboratory information system as shown in Table 1.

Table 1 Positivity rate of suspected syphilis samples using SD Bioline test

We then randomly selected 100 samples from the syphilis suspected specimens that tested reactive and non-reactive (1:1) on SD Bioline which were compared with the reference standard, Architect syphilis TP. We used the available electronic laboratory information system to confirm the clinical suspicion for syphilis. Samples had been collected through venipuncture into plain glass vacutainers containing no clot activators, anticoagulants, preservatives or separator materials which and were stored at 2–8 °C for a minimum of 45 min before centrifugation to extract serum for testing.

Test methods

The SD Bioline syphilis 3.0 (Standard diagnostics, INC. Korea) is a solid phase immunochromatographic assay for the qualitative detection of antibodies of all isotypes [Immunoglobulin G (IgG), Immunoglobulin M (IgM), Immunoglobulin A (IgA)] against TP. It is pre-coated with recombinant TP antigens that bind to the TP antibodies, if present in the patient sample to produce a visible line. In this study, SD Bioline syphilis 3.0 is the index test.

The Architect syphilis TP (Abbott GmbH & Co. KG, Germany) is a two-step chemiluminescent microparticle immunoassay (CIA or CMIA) for the qualitative detection of IgM and IgG antibodies to TP intended to aid in the diagnosis of syphilis infection and as a screening test in blood donors to prevent transmission of TP to recipients. The method and principle of analysis are described in detail elsewhere [8]. Architect syphilis TP was used as our reference standard.

Bias

Since the sample size was not calculated based on any hypotheses but selected based on the availability of reagents, we most likely introduced selection bias by selecting 50 positives and 50 negative samples. However, the selection of these samples was random to minimize selection bias.

Statistical methods

Descriptive statistics was used to summarize data while Chi square or fishers exact test compared results for the two diagnostic testing methods. A concordance test assessed the degree of agreement between the two tests and receiver operating curve (ROC) to evaluate the benefit of using the index test.

Reporting guidelines

We used the Standards for Reporting of Diagnostic Accuracy Studies (STARD) [9].

Results

Total number of syphilis suspected samples collected and tested between March and May were 781 (Table 1) of which 116 (14.9%) tested positive with SD Bioline syphilis rapid kit. There was a drop, in positivity rate for the month of May compared to the first 2 months.

50 samples that tested negative and positive respectively, on SD Bioline syphilis rapid kit were retested on Architect and the diagnostic tests then evaluated (Table 2).

Table 2 False positivity and false negative rates of SD Bioline compared with architect syphilis test

As shown in Table 2, the false positive rate for SD Bioline was 14.8% with a sensitivity of 91.3% while the false negative rate was at 8.7% with a specificity of 85.2%. There was a significant difference between the two tests p < 0.0005. Positive and negative likelihood ratios of 6.2 and 0.1 were reported respectively (data not shown).

The positive and negative predictive values were 84% and 93% at prevalence of 54% (expected). However, using actual point prevalence of 15.3%, the positive and negative predictive values were 53.7% and 46.9%.

Roc curve for SD Bioline against the Architect test

We performed a receiver operator curve (ROC) analysis (graph not shown) yielding an area under the curve of 0.88, (standard error, 0.032; 95% confidence interval [0.81, 0.95] p < 0.001).

We also performed a concordance test (Table 3 below) to assess the degree of agreement between the two tests.

Table 3 Cohen’s kappa (ĸ) results showing a good degree of concordance between SD Bioline and Architect tests

Discussion

The specificity (85.3%) and sensitivity (91.3%) of SD Bioline against the Architect in our study were significantly low especially that SD Bioline and the Architect sensitivity and specificity compared to several non-treponemal tests can go as low as 95% and 98% respectively [10]. However, in another study, the sensitivity of SD Bioline was comparable (92%) [11], moreover a study conducted in Tanzania found the test to be 79% sensitive and 96% specific [12].

The marginal distributions of test ratings did not indicate prevalence or bias problems, suggesting that Cohen’s kappa was an appropriate index of inter-rater reliability, so we performed an inter-rater reliability test using Cohen’s kappa (ĸ) to determine if there was agreement between the two diagnostic tests in our study. The results (88%, ĸ = 0.76; p < 0.0005) obtained indicate substantial agreement [13]. An almost perfect agreement (ĸ = 0.81–1.00) is the best every lab would want to achieve. However, the difference could be explained by the technicality involved in the performance of the two tests, the architect being fully automated, hence not prone to a lot of errors and expert interpretation as compared to the immunochromatographic method used in the SD Bioline syphilis 3.0 which is prone to pre- and -analytical errors. Nonetheless, the scope of our study could not allow us to determine the real difference.

The clinical implications of our results in the diagnosis of syphilis can only be assessed with additional clinical information. It remains a standard that when interpreting laboratory results for syphilis, we must consider the clinical stage aided by patient history and physical examination. Without these, erroneous diagnosis and bad management decisions is the consequence [4, 14].

The laboratory implication from the results we obtained warrants a follow up study to investigate and determine the efficiency and application of the chemiluminescence immunoassay and immunochromatographic assay in syphilis diagnosis as it is difficult to prove that positive or negative results obtained using these platforms are necessarily false [1, 15] hence the need for patient medical history including previous treatment for syphilis, and history of sexual risk factors [1, 2]. Additionally, we recommend adding a non-treponemal test like RPR and possibly implement reverse algorithm in syphilis diagnosis. The use of both non-treponemal and treponemal tests for diagnosis is warrantable for laboratories [1]. It also means that the laboratory needs to obtain complete clinical information for accurate interpretation of results to guide the clinician in managing the patients appropriately [16]. The test methods involved in our study are treponemal tests. It is well known that these tests can remain positive almost for life in previously infected individuals regardless of treatment [1] while antibodies measured in non-treponemal tests do disappear after some years [17]. It is therefore, vital, to use both treponemal and non-treponemal tests, to differentiate previous infection from current [17]. However, we also know that a confirmed positive serological test result does not differentiate disease stage and we may not tell if the infection is past or current in some cases [17]. Given the limitations of TP tests, we can still use them to make accurate diagnosis aided by clear patient history and evidence from physical examination [17].

For primary syphilis, the recommended tests to perform include direct examination, non-treponemal tests, and treponemal tests and since there is no single optimal test, a combination of these is required to aid in the diagnosis [1, 2]. Detection of TP in lesions is definitive evidence of syphilis but a negative result does not rule out syphilis [18]. PCR-based tests have a high reliability [1, 18]. In the first 2–3 weeks, serology may not be positive in some cases, and in early primary syphilis, treponemal tests maybe recommended [19]. When there is evidence of a genital ulcer at physical examination and a non-treponemal test is reactive, this may not necessarily indicate primary syphilis [17] and in such cases a repeat test over a two to 12 week period may be required to rule out syphilis [17]. In secondary syphilis, the use of direct physical examinations, and use of both non-treponemal tests and treponemal tests are still recommended [17, 19]. In persons with a history of syphilis, a fourfold increase in titer when using quantitative methods provides presumptive diagnosis of secondary syphilis [17]. In latent syphilis, as in all stages, it is recommended to use both treponemal and non-treponemal tests [17]. However, non-treponemal tests are reactive in early latent syphilis though the sensitivity declines over time [1, 17]. Prevalence information of syphilis in a population is cardinal for interpretation, as in low prevalence populations, false-positive results may be common with both treponemal and non-treponemal tests [17].

As a general rule, a reactive treponemal test in the absence of a reactive non-treponemal test may require confirmation [17]. For tertiary syphilis, non-treponemal and treponemal tests are recommended [15, 17, 19]. In this stage approximately 30% may not be reactive using non-treponemal tests, while treponemal tests are reactive almost always [17]. Therefore, we think treponemal tests should always be considered [17].

Conclusion

The specificity and sensitivity of the immunochromatographic assay, SD Bioline syphilis 3.0 against the chemiluminescent immunoassay, Architect syphilis TP was 85.3% and 91.3% respectively with substantial agreement between the two test methods. However, we recommend further comprehensive study with a larger sample size and clinical details to ascertain the validity of our findings. We recommend using both treponemal and non-treponemal tests to aid diagnosis of syphilis.

Limitations

Because we did not have access to clinical information, our study had limited variables needed for a comprehensive comparison. For example, we needed to compare our results to clinical findings and as well, compare with a non-treponemal test in order to provide an accurate conclusion on the implications of the false positive and false negative rates.

Our cohort was not well characterized to draw useful conclusions with clinical and laboratory implications. However, we want to use the lessons from our findings to propose for a well characterized follow-up study.

False-positive reactions can occur with treponemal tests but this is less common than with nontreponemal tests [17]. Examples of conditions were false positive tests are reported include but not limited to advancing age, brucellosis, cirrhosis, hyperglobulinemia, malaria, pregnancy, systemic lupus erythematosus et cetera [17]. Our study lacked information on these factors that have implications on the interpretation of TP laboratory results.

Abbreviations

CDC:

Centres for Disease Control and Prevention

CIA OR CMIA:

chemiluminescence immunoassay

IgA:

Immunoglobulin A

IgG:

Immunoglobulin G

IgM:

Immunoglobulin M

ROC:

receiver operating curve

RPR:

rapid plasma reagin

STARD:

Standards for Reporting of Diagnostic Accuracy Studies

TP:

Treponemal pallidum

References

  1. Morshed MG, Singh AE. Recent trends in the serologic diagnosis of syphilis. Clin Vaccine Immunol. 2015;22:137–47.

    Article  CAS  Google Scholar 

  2. Centers for Disease Control and Prevention (CDC). Discordant results from reverse sequence syphilis screening–five laboratories, United States, 2006–2010. MMWR Morb Mortal Wkly Rep. 2011;60:133–7.

    Google Scholar 

  3. Jonckheere S, Berth M, Esbroeck MV, Blomme S, Lagrou K, Padalko E. Evaluation of different confirmatory algorithms using seven treponemal tests on Architect Syphilis TP-positive/RPR-negative sera. Eur J Clin Microbiol Infect Dis. 2015;34:2041–8.

    Article  CAS  Google Scholar 

  4. Workowski KA, Berman S, Centers for Disease Control and Prevention (CDC). Sexually transmitted diseases treatment guidelines, 2010. MMWR Recomm Rep. 2010;59:1–110.

    PubMed  Google Scholar 

  5. Binnicker MJ. Which algorithm should be used to screen for syphilis? Curr Opin Infect Dis. 2012;25:79–85.

    Article  Google Scholar 

  6. Lee J-H, Lim CS, Lee M-G, Kim H-S. Evaluation of a rapid immunochromatographic treponemal antibody test comparing the Treponema pallidum particle agglutination assay. J Clin Lab Anal. 2015;29:383–6.

    Article  CAS  Google Scholar 

  7. Binnicker MJ, Jespersen DJ, Rollins LO. Treponema-specific tests for serodiagnosis of syphilis: comparative evaluation of seven assays. J Clin Microbiol. 2011;49:1313–7.

    Article  CAS  Google Scholar 

  8. Park Y, Park Y, Joo SY, Park MH, Kim H-S. Evaluation of a fully automated treponemal test and comparison with conventional VDRL and FTA-ABS tests. Am J Clin Pathol. 2011;136:705–10.

    Article  Google Scholar 

  9. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6:e012799.

    Article  Google Scholar 

  10. Seña AC, White BL, Sparling PF. Novel Treponema pallidum serologic tests: a paradigm shift in syphilis screening for the 21st century. Clin Infect Dis. 2010;51:700–8.

    Article  Google Scholar 

  11. Mehra B, Bhattar S, Saxena S, Rawat D, Bhalla P. Evaluation of SD BIOLINE syphilis 3.0 for rapid diagnosis of syphilis: report from a regional sexually transmitted infection reference laboratory in North India. J Lab Physicians. 2016;8:36–40.

    Article  CAS  Google Scholar 

  12. Aboud S, Lyamuya EF, Sufi J, Haule E. Comparison of teh performance of SD Bioline Syphilis 30 assay with the RPR test for the syphilis screening in Dar es Salaam. Tanzania. Tanzania Med J. 2006;21:6–7.

    Google Scholar 

  13. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22:276–82.

    Article  Google Scholar 

  14. Larsen SA, Steiner BM, Rudolph AH. Laboratory diagnosis and interpretation of tests for syphilis. Clin Microbiol Rev. 1995;8:1–21.

    CAS  PubMed  PubMed Central  Google Scholar 

  15. Mattei PL, Beachkofsky TM, Gilson RT, Wisco OJ. Syphilis: a reemerging infection. Am Fam Physician. 2012;86:433–40.

    PubMed  Google Scholar 

  16. Soreng K, Levy R, Fakile Y. Serologic testing for syphilis: benefits and challenges of a reverse algorithm. Clin Microbiol Newslett. 2014;36:195–202.

    Article  Google Scholar 

  17. Ratnam S. The laboratory diagnosis of syphilis. Can J Infect Dis Med Microbiol. 2005;16:45–51.

    PubMed  PubMed Central  Google Scholar 

  18. Tong M-L, Lin L-R, Liu G-L, Zhang H-L, Zeng Y-L, Zheng W-H, et al. Factors associated with serological cure and the serofast state of HIV-negative patients with primary, secondary, latent, and tertiary syphilis. PLoS ONE. 2013. https://doi.org/10.1371/journal.pone.0070102.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Brown DL, Frank JE. Diagnosis and management of syphilis. Am Fam Physician. 2003;68:283–90.

    PubMed  Google Scholar 

Download references

Authors’ contributions

SKM and BMH conceived the study. SKM, MM, and BMH conducted the data collection. SKM, MM, AK, YC, PT, BMH contributed to data analysis and led the writing of the manuscript. SKM is the principal investigator and guarantor. BMH is the senior author. All authors read and approved the final manuscript.

Acknowledgements

We are very grateful to all Laboratory personnel and the office of the Senior Medical Superintendent at Livingstone Central hospital for their support and enthusiasm for research projects.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

All data generated or analysed during this study are included in this published article. For other data, these may be requested through the corresponding author.

Consent to publish

Not applicable.

Ethics approval and consent to participate

This study is part of an approved larger study being conducted at Livingstone Central hospital. Ethical approval was obtained from the University of Zambia Biomedical Research Ethics Committee (Assurance No. FWA00000338 IRB00001131 of IORG0000774) and permission to conduct the study at Livingstone Central hospital was granted by the Hospital Administration.

Funding

This protocol has not received any grant from any funding agency in the public, commercial or not-for-profit sectors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sepiso K. Masenga.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Masenga, S.K., Mweemba, M., Kachele, A. et al. Lessons from a comparison of immuno-chromatographic and chemiluminescent micro-particle immunoassay in the diagnosis of syphilis. BMC Res Notes 11, 701 (2018). https://doi.org/10.1186/s13104-018-3808-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-018-3808-5

Keywords