Skip to main content

Quality of observational studies in prestigious journals of occupational medicine and health based on Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: a cross-sectional study

Abstract

Objective

The present study applied the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement to observational studies published in prestigious occupational medicine and health journals.

Results

A total of 60 articles was evaluated. All sub-items were reported in 63.74% (95% confidence interval [CI], 56.24–71.24%), not reported in 29.70% (95% CI, 20.2–39.2%), and not applicable in 6.56% (95% CI, 4.86–8.26%) of the studies. Of the 45 sub-items investigated in this survey, eight were reported 100% of the time, 13 were addressed in more than 90% of the articles, 22 were included in more than 75% of the studies, and 27 sub-items were applied in more than 50% of the articles published in the journals included in this study.

Introduction

Observational studies have an important role in researching the benefits and harms of medical interventions [1]. The results of these studies should be reported as transparently as possible “so that readers can follow what was planned, what was done, what was found, and what conclusions were drawn” [2]. The credibility of a research depends on a critical assessment by others, in the study design, conduct, and analysis of it [3]. In order to assess the strengths and weaknesses of the evidence from the observational studies von Elm et al. designed in 2007 a 22-item checklist to assist with clear reporting of observational studies called: the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement [2]. This checklist includes a description of methodological items and instructions on how to use them to transparently report observational studies. Several extensions of these statements have been published with additional recommendations for specialized fields of research—for example, the Strengthening the Reporting of Observational Studies in Epidemiology for Newborn Infection (STROBE-NI) [4] and Strengthening the Reporting of Observational Studies in Epidemiology—nutritional epidemiology (STROBE-nut) [5].

To our knowledge, the quality of reporting in occupational observational studies has not been assessed. Observational study designs are often used in studies published in occupational medicine and health journals. Therefore, the present study investigated the quality of occupational observational studies reporting post-STROBE statements. Application of these recommendations by upcoming observational studies in occupational health and medical journals, will increase the value of new data and avoid wasted research.

Main text

Methods

Journal identification

We conducted a cross-sectional study and selected four top occupational medicine and health journals with high-impact factors (IFs) based on the Information Sciences Institute (ISI) in the Web of Knowledge (http://www.webofknowledge.com, date of access 14 Aug 2017) as of 2016, including journals in the first quartile in category (Q1) of scientific journal ranking (http://www.scimagojr.com, date of access 14 Aug 2017) among the most prestigious and impactful occupational medicine and health journals indexed in international databases. In addition, we checked the `Instruction for authors’ section of the websites for each of the included journals to determine if they contained the STROBE statement for authors or endorsed it as a guideline for reporting observational research articles (Table 1).

Table 1 Top four occupational medicine and health journals and impact factor and STROBE endorsement

Study search and selection

We first evaluated the archives from January 1 until July 19, 2017, to retrieve observational studies (cohort, case–control, and cross-sectional studies) published in each of the selected journals. During this period, we identified enough cohort and cross sectional studies, but did not find enough case–control studies; thus, only for this type of study, we expanded the search range from January 1, 2016, to July 19, 2017. The search revealed 188 observational articles.

Second, the types of articles were classified according to the journal in which they were published. Next, a specific number was assigned to each selected article and, based on random numbers table, we randomly selected 15 observational (five each cohort, case–control, and cross sectional) studies published in each of the four prestigious occupational medicine and health journals. Accordingly, we enrolled 60 observational articles in our study, which were randomly assigned to two reviewers without blinding the name of the journal or the authors of the articles. The reviewers independently made decisions regarding the number of items from the STROBE check-list which were addressed in the selected studies.

Reliability and validity adherence

To check the judgment of two reviewers regarding the quality of observational studies, we conducted a pilot study as follows: first, among the articles that were previously retrieved, an article was selected randomly and the strengths and weaknesses of its evidence were assessed based on the predetermined checklist of items. Any disagreement over the items was discussed to reach the same interpretation of the checklist in order to increase the reliability between the two reviewers.

Second, two articles were selected randomly from among all the selected studies for the two reviewers to evaluate the checklist items based on three choices for each sub-item (reported/not reported/not applicable). This work was done in order to calculate an indicator of the reliability between two reviewers, the KAPPA statistic, in evaluating the items of the STROBE checklist in the selected articles. The KAPPA statistic was calculated according to the formula below:

$${\text{KAPPA}} = \frac{{({\text{Percent}}\;{\text{agreement}}\;{\text{observed}}) - ({\text{Percent}}\;{\text{agreement expected}}\;{\text{by}}\;{\text{chance}}\;{\text{alone}}\;)}}{{100\% - ({\text{Percent}}\;{\text{agreement}}\;{\text{expected}}\;{\text{by}}\;{\text{chance}}\;{\text{alone}})}} =$$

Landis and Koch suggested that a Kappa greater than 0.75 represents excellent agreement beyond chance, a Kappa below 0.40 represents poor agreement and a Kappa of 0.40–0.75 represents intermediate to good agreement [6]. Based on Additional file 1: Table S1 and Additional file 2: Table S2, the Kappa statistic between reviewers A and B in this study was 53%

$${\textit{KAPPA}} = \frac{{76.7 - 50.86\% }}{{100 - 50.86\% }} = 0.53.$$

In order to assess the validity of the reviewers’ judgments in this cross-sectional study, reviewers A (J. Ah) and B (K. Mo) read and scored the included observational articles and independently made decisions regarding the quality of reporting in each observational study. Any disagreements were resolved by adjudication with the third (I. Mo) and fourth authors (B. Ma).

Data analysis

Statistical analyses were conducted using IBM SPSS Statistics for Windows, version 20.0 (IBM Corp; Armonk, NY, USA). The absolute frequencies and percentages of each sub-item in the selected articles were addressed. The total percentage for all sub-items (1–22) was reported (Table 2).

Table 2 Percentage of items in the STROBE checklist which were addressed in cohort, case control, and cross-sectional studies published in four top scientific occupational journals in 2017

Results

Figure 1 shows the flow diagram of the search process and the numbers of observational articles included at each stage of the study published in prestigious scientific occupational medicine and health journals in 2016–2017. During the search process, of 281 original articles, 93 were not observational studies and were excluded from the study. Finally, we selected 60 observational studies from four prestigious scientific occupational medicine and health journals, including Scand J Work Environ Health, Occup Environ Med, J Occup Environ Med, and Am J Ind Med.

Fig. 1
figure 1

The overview of study design. STROBE checklist of items was addressed in prestigious scientific occupational medicine and health journals, 2016–2017

The absolute frequencies and percentages of items and sub-items addressed by these studies are summarized in Table 2. The sub-items were reported in 63.74% (95% confidence interval[CI], 56.24–71.24%), not reported in 29.70% (95% CI, 20.2–39.2%), and not applicable in 6.56% (95% CI, 4.86–8.26%) of the cohort, case–control, and cross sectional studies. This table also shows that, of the 45 sub-items investigated in this survey, collectively, eight sub-items were reported 100% of the time, 13 sub-items were addressed in more than 90% of the articles, 22 sub-items were included in more than 75% of the studies, and 27 sub-items were applied in more than 50% of the articles assessed from the journals included in this study.

Additional file 3: Figure S3 shows the percentages of adequately reported STROBE sub-items in articles published in the top four occupational medicine and health journals. In this figure, sub-items including 2-Introduction, 7a-Outcomes, 8a-Sources of data, 12a-Control for confounding, 15-N of outcome events, 18-Key results, 20a-Interpretation, and 20b-Similar studies were all reported in the articles published in the four prestigious journals included in this study.

Additional file 4: Figure S4 shows the percentages of STROBE sub-items not reported in articles published in the top four occupational medicine and health journals. Collectively, the five sub-items with the lowest reporting in the articles were 12f-Sensitivity analyses, 13b-Non-participation, 17-Other analyses, 14b-N of participants with missing data, and 12c-Interactions.

Additional file 5: Figure S5 shows that among 45 sub-items in the STROBE statement, those such as 12e-Loss to follow-up, 6d-N of exposed and unexposed, and 6c-matching criteria were the most common not applicable items in the articles included in the present study.

Discussion

The STROBE statement provides valuable recommendations for all authors when reporting the results of analytical observational studies, supports editors and reviewers when considering these articles for publication, and helped readers to critically appraise published articles [2]. In total, approximately 63.47% of the items and sub-items in the STROBE checklist were reported in cohort, case–control, and cross-sectional studies published in the four journals of occupational medicine and health included in the present study, 11 years after the dissemination of the STROBE statement. However, numerous observational studies are published in less fastidious peer-reviewed occupational medicine and health journals. Thus, it is expected that the quality of reporting of such studies is poorer than that reported in the present survey, although the results of the present study indicate that even this quality is not sufficient. To our knowledge, this study is the first to assess the quality of reporting of observational studies in occupational medicine and health literature; however, similar work has been performed in other disciplines; for example, Hendriksma et al. [7]. in 2016 con-ducted a similar study to evaluate the quality of reporting of observational studies in otorhinolaryngology based on the STROBE statement. They reported that the articles in the top five general medical journals reported a mean of 69.2% (95% CI 65.8 ± 72.7%) of items, compared to 51.4% (95% CI 47.7 ± 55.0%) in the top five ear, nose, and throat journals. These results are similar to the mean of 63.74% (95% CI 56.24–71.24%) obtained in our study. Jeelani et al. [8]. conducted a similar study in 2014 to assess the quality of reporting of cross-sectional studies published in the Indian Journal of Community Medicine by evaluating the extent to which they adhered to the STROBE statement. In this study, the most frequently reported checklist items included the summary of what was done and what was found in the abstract, background/rationale, objectives, setting, outcome data, key results in the discussion, and interpretation of results. In our study (Additional file 2: Table S2) the most frequently reported STROBE checklist items were the explanation of the scientific background and rationale for the investigation being reported; clearly defining all outcomes; providing sources of data; and describing all statistical methods, including those used to control for confounding; reporting the numbers of outcome events or summary measures; summarizing key results with reference to study objectives; providing a cautious interpretation of results considering the objectives, and explaining the results of similar studies. The findings in our study are consistent with those reported in the previous study [8, 9].

In 2016, Agha et al. [9] assessed the compliance of observational studies in plastic surgery using the STROBE statement checklist. The average STROBE score in his study was 12.4 (range 2–20.1) with a standard deviation of 3.36. This mean in plastic surgery articles is not satisfactory and lower than the average obtained from the occupational medicine and health journals evaluated in the present study.

The results of this study reveal that the quality reporting of observational studies published in the most prestigious occupational medicine and health journals is yet not clear and desirable enough. Thus, this issue should be the focus of the both authors’ and editors’ special attention when reporting and/or reviewing the reports of observational studies.

Limitations

  • First, the random selection of cohort, case–control, and cross sectional studies from prestigious occupational medicine and health journals may result in a selection bias in the results.

  • Second, the limited number of articles selected and evaluated in this survey, may in-crease the possibility of random error.

  • Third, the quality and accuracy of the STROBE sub-items addressed in the selected articles depended primarily on the judgment of the reviewer, which increased the probability of information bias in the results.

Abbreviations

STROBE:

The Strengthening the Reporting of Observational Studies in Epidemiology

IFs:

impact factors

ISI:

The Information Sciences Institute

Q1:

in the first quartile in category

Scand J Work Environ Health:

Scandinavian Journal of Work, Environment and Health

Occup Environ Med:

Occupational and Environmental Medicine

J Occup Environ Med:

Journal of Occupational and Environmental Medicine

Am J Ind Med:

American Journal of Industrial Medicine

References

  1. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ Br Med J. 1996;312(7040):1215.

    Article  CAS  Google Scholar 

  2. Von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, Initiative S. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495–9.

    Article  Google Scholar 

  3. Jüni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ Br Med J. 2001;323(7303):42.

    Article  Google Scholar 

  4. Fitchett EJ, Seale AC, Vergnano S, Sharland M, Heath PT, Saha SK, Agarwal R, Ayede AI, Bhutta ZA, Black R. Strengthening the Reporting of Observational Studies in Epidemiology for Newborn Infection (STROBE-NI): an extension of the STROBE statement for neonatal infection research. Lancet Infect Dis. 2016;16(10):e202–13.

    Article  PubMed  Google Scholar 

  5. Lachat C, Hawwash D, Ocké MC, Berg C, Forsum E, Hörnell A, Sonestedt E, Wirfält E, Åkesson A, Kolsteren P. Strengthening the Reporting of Observational Studies in Epidemiology–nutritional epidemiology (STROBE-nut): an extension of the STROBE statement. Nutrition bulletin. 2016;41(3):240–51.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Guidotti LG. Epidemiology. 4th ed. Philadelphia: Saunders; 2008.

    Google Scholar 

  7. Hendriksma M, Joosten MH, Peters JP, Grolman W, Stegeman I. Evaluation of the Quality of Reporting of Observational Studies in Otorhinolaryngology-Based on the STROBE Statement. PLoS ONE. 2017;12(1):e0169316.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Jeelani A, Malik W, Haq I, Aleem S, Mujtaba M, Syed N. Cross sectional studies published in Indian journal of community medicine: evaluation of adherence to the strengthening the reporting of observational studies in epidemiology statement. Ann Med Health Sci Res. 2014;4(6):875–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Agha RA, Lee S-Y, Jeong KJL, Fowler AJ, Orgill DP. Reporting quality of observational studies in plastic surgery needs improvement: a systematic review. Ann Plast Surg. 2016;76(5):585–9.

    Article  CAS  PubMed  Google Scholar 

Download references

Authors’ contributions

Conceived, development and design of methodology in the study: IM and JA. Helped to data collection: JA-A, KM, JA and BM. Analyzed the data: JA-A, KM, JA and BM. Wrote the paper: IM and JA. All authors read and approved the final manuscript.

Acknowledgements

We thank Urmia University Medical Sciences for supporting this study.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The datasets generated during the current study available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Funding

This study was supported by a Grant (2183) and funding from the Urmia University Medical Sciences. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Iraj Mohebbi.

Additional files

Additional file 1: Table S1.

Percent agreement between reviewers A and B.

Additional file 2: Table S2.

Percent agreement between reviewers A and B expected by chance alone.

Additional file 3: Figure S1.

Percentages of adequately reported STROBE sub-items in articles published in the top four occupational medicine and health journals.

Additional file 4: Figure S2.

Percentages of STROBE items not adequately reported in articles published in the top four occupational medicine and health journals.

Additional file 5: Figure S5.

Percentages of not applicable STROBE items in articles published in the top four occupational medicine and health journals.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghazadeh-Attari, J., Mobaraki, K., Ahmadzadeh, J. et al. Quality of observational studies in prestigious journals of occupational medicine and health based on Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: a cross-sectional study. BMC Res Notes 11, 266 (2018). https://doi.org/10.1186/s13104-018-3367-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-018-3367-9

Keywords