What time-lag for a retraction search on PubMed?

Abstract

Background

To investigate fraud and errors, scientists have studied cohorts of retraction notices. These researches have been performed using a PubMed search on publication type “retraction of publication” which retrieves the notices of the retractions. We assessed the stability of the indexation of retraction notices over a 15-month period and what was the time-lag to get stability.

Findings

A search on notices of retraction issued in 2008 was repeated every 3 months during 15 months from February 2011. The first search resulted in 237 notices of retraction. Throughout the study period, 14 discrepancies with the initial search were observed (6%). We found that the number of retraction notices became stable 35 months after the retraction.

Conclusions

The time-lag observed in this study has to be taken into account when performing a PubMed search.

Findings

To investigate fraud and errors, scientists have studied cohorts of retraction notices[1–6]. These researches have been performed using a PubMed search on publication type “retraction of publication” which retrieves the notices of the retractions. The ability to find all retraction notices published in a given period is essential for these researches, and these studies rely on the fact that all retraction notices are identified as such in PubMed records. If the notification in the journal is labeled as a retraction or withdrawal, NLM will index it as a retraction (http://www.nlm.nih.gov/pubs/factsheets/errata.html). However, indexation process could also be prone to errors. Accuracy of indexation could not be assessed since it would require to have access to the full population of retraction notices, which is actually unknown. We therefore decided to assess the stability of the indexation of retraction notices over a 15-month period and what was the corresponding time-lag.

An initial search on the publication type “retraction of publication” issued in 2008 was performed[6] (“retraction of publication”[Publication Type] AND (“2008”[PDAT]: “2008”[PDAT])). We then repeated it every 3 months during 15 months, from February 2011. Each search was compared to the previous one to find discrepancies, which were classified as: newly identified retraction notice (not indexed in the former search) or change in the retraction’s authors. We tried to identify the reason for these changes. These classifications were not defined a priori.

The first search performed in February 2011 resulted in 237 notices of retraction published for the year 2008. Throughout the study period, 14 discrepancies with the initial search were observed (6%).

Firstly, 9 notices were newly identified, the last appearing 9 months after the first search (Table 1). Among these, the word “retraction” was present in the title for 6 at the time of the search. Concerning the explanation for the late indexation, in 7 cases, although the e-publication date was in 2008, the publication date of the retraction notice was in 2011, certainly leading to the update of the PubMed record with re-indexation (see Table 2 for an example). In one case, a correction to the retraction notice was issued in 2011, certainly leading to the modification of the initial indexation of the notice. For 1 case, we could not find any explanation for the late indexation.

Table 1 Quaterly PubMed searches during 15 months on 2008 retraction notices

Table 2 Example of an e-publication date in 2008 and a publication date in 2011 and the corresponding Medline indexation*

Secondly, a total of 5 discrepancies on the author list was observed. They consisted in the deletion of the author list initially available (Table 1). All these modifications occurred in notices which were newly identified during our study.

We found that the number of retraction notices became stable in November 2011 for the retraction notices of the year 2008 i.e., 35 months after. This result shows that retraction notices, despite being a very specific entity, are not always indexed as “retraction of publication” in PubMed. However, as raised by Ivan Oransky (http://retractionwatch.com/), there is no other available database for retractions.

The time-lag observed in this study has to be taken into account when performing a PubMed search and a time-lag of at least 3 years should be respected between the time of the search and the year of interest.

Errors in indexation were corrected when the PubMed record had to be updated (publication, erratum), consequently we cannot ascertain that all retraction notices are indexed as such. Therefore, to ease indexation process, retraction notices titles should at least include the word “retraction” as recommended by the Committee on Publication Ethics (COPE)[7]. Furthermore, the use of a standard retraction form would be very useful as it could help to standardize the title as well as the way of presenting authors for retractions[6].

Authors’ information

ED is a senior researcher (PhD), LH is a senior researcher (PharmD, PhD), HM is a senior researcher (MD).