Skip to main content

Sensitivity and specificity of a brief scale to evaluate psychological violence at work in Peruvian health professionals

Abstract

Objective

Most studies in Latin America that evaluate psychological violence at work (PVW) focus on measuring occurrences of PVW. However, the discriminative validity and randomness of instruments used for evaluating incidents of PVW that are generated by agents internal to the workplace in the Peruvian health sector have not yet been studied. The objective of this study was to determine the sensitivity and specificity of the Scale of Psychological Violence in Health Professionals (SVP-Health) in the Peruvian population. For this purpose, a cross-sectional study based on the two-stage administration of guided surveys and ROC (receiver operating characteristic) curve analysis was performed.

Results

The study included 188 professionals from ten care centres in Peru. The professionals were divided into two groups of 94 subjects: those who had experienced PVW and those who had not. The average age of the participants was 36.8 ± 10.5 years; their work experience ranged between one and 35 years; and 59% of the sample were women. According to an analysis based on the ROC curve, (a) there was a significant area under the curve (AUC = 0.899) with adequate randomness; and (b) the SPV-Health adequately distinguished subjects with PVW from those without PVW (89% versus 94%).

Introduction

Psychological violence at work (PVW) refers to an intentional action in which workers are verbally assaulted, threatened and/or humiliated during their professional practice [1, 2] in ways that do not involve physical violence [3]. PVW is exacerbated in the health sector due to the precariousness of working conditions, which commonly involve psychosocial risk factors [4,5,6]. There are no differences in the rates of PVW between developed and underdeveloped countries, between health groups, or between stable and temporary work conditions [7,8,9,10]; however, at the individual level, the effects of PVW can vary according to the individual’s resilience [6, 11].

At the global level, there is a considerable variety of instruments available for evaluating and monitoring the prevalence and impact of PVW [3, 10], but few studies have examined Peruvian health professionals’ experiences with PVW perpetrated by internal agents (that is, superiors or colleagues) [9, 12] and PVW related to interpersonal conflicts or demands. This is partly because some analytical tools, for example, structured formats or the aggressive behaviour scale [10, 13, 14], have been validated for Spain, Mexico, Colombia, Ecuador, Chile and Bolivia but not for the Peru.

Specifically, in Bolivia, Ecuador and Chile, although the convergent validation of the General Health Questionnaire (12 items) has been established and diagnostic scales have been created to measure violence and psychological harassment (including external violence) in workers in the service, industry, commerce and education sectors, the sensitivity and specificity of these diagnostic scales have not been determined [15, 16].

The instruments used in Peru to evaluate PVW in different health care settings have limitations, since their usefulness and the classifications they propose have not been established and, consequently, they do not meet the standards of use for psychological tests [17, 18].

Due to the limitations of the existing measures, it is necessary to develop an instrument with sufficient utility, discriminative capacity and randomness to classify workers exposed to PVW [19]. In addition, it is necessary to standardize the criteria for the interpretation of the results to identify, evaluate and compare the prevalence of PVW according to location (Lima versus other cities), work activity (nursing, medicine and others) and employment status (stable versus temporary) [12, 17] to verify corrective actions and establish baselines in matters of mental health at work. The lack of tools with reference scales and discriminative capacity hinders standardized evaluations of PVW.

This report complements the Scale of Psychological Violence in Health Professionals (SPV-Health) assessment and allows us to answer the following question: What is the best cut-off point for classifying cases with and without psychological violence according to the SPV-Health? Our objective is to determine the sensitivity and specificity of the SPV-Health in the Peruvian population.

Main text

Methods

This is a cross-sectional study based on the administration of guided surveys in two stages: (a) the classification of groups of equal size comprised of participants with and without PVW according to the external criterion of job satisfaction (JS), which was evaluated using an overall job satisfaction scale (OJS); and b) evaluation of the sensitivity and specificity of the SPV-Health in both groups (participants with and without PVW). The surveys were conducted between December 2019 and February 2020. To determine the sample size, the minimum recommendations established for external validation and receiver operating characteristic (ROC) curves were adopted: groups of n ≥ 176–200 participants [20, 21] with similar sex distributions [12]. No calculation was required.

The inclusion criteria for the participants were (a) complete responses to both assessment instruments (the OJS and SPV-Health) and (b) having worked at least 6 months before the time of the evaluation. The exclusion criterion was less than 2 years of work experience [12].

Instruments

SPV-Health (short scale used to evaluate psychological violence in health professionals)

For this study, a scale was validated that comprises 22 items referring to three types of violence in the context of Peruvian health care settings: isolation, intimidation and discreditation [22]. Each item is rated on a 4-point Likert scale (ranging from 1, “never” to 4, “always”) (see Additional file 1). The scale showed good reliability and validity in a population of 316 health professionals [22]. Its properties are as follows:

  1. a)

    Good content validity, evaluated with the judgement of five experts (doctors, nurses and teaching psychologists), with a Kendall coefficient (W = 0.509; p < 0.026) that exceeded the established minimum values [23].

  2. b)

    Appropriate construct validity, with factors that explain up to 54.3% of the total variance, which exceeds the minimum (50%) [24]; confirmatory factors of moderate adjustment; root mean square error of approximation and standardized residual of the mean square root above the acceptability limit; and acceptable comparative fit and Tucker–Lewis indices (0.60).

  3. c)

    Good convergent validity with the OJS (r = − 0.769; p < 0.0003).

  4. d)

    Good internal consistency reliability, with a global Cronbach’s alpha coefficient of α = 0.803 and an item Cronbach’s alpha of α = 0.800 (acceptable values are α = 0.70–0.80) [18]. Reliability was assessed in terms of response stability; there were significant differences between the first and second applications, which were separated by a 4-month interval (p < 0.008).

Overall Job Satisfaction Scale (OJS)

The OJS consists of 15 items grouped into two subscales: (a) intrinsic job satisfaction, which is related to task-based factors such as recognition, responsibility and promotion (7 items); and (b) extrinsic satisfaction, which is related to satisfaction with the organization, schedules, wages and physical conditions of the work (8 items). Each item is rated on a 7-point Likert scale (1, “very dissatisfied” to 7, “very satisfied”). The scale presented good reliability and validity in 518 Spanish nurses [14], with an overall reliability of alpha 0.75. Its construct reflects experiences and emotional responses at work [15].

Procedure

The study subjects were informed about the objectives of the research, and their voluntary participation was recorded through a letter of consent in which the participants’ anonymity and the confidentiality of the data were guaranteed. The simultaneous collection of data was carried out by organizational psychologists and health professionals.

Statistical analysis

To evaluate the sensitivity and specificity of the SPV-Health in groups of participants with and without PVW, a ROC curve analysis was performed, and the area under the curve (AUC), the standard error (SE) and the cut-off point that maximizes the sensitivity–specificity of the instrument were determined [20, 25]. To divide the results into three levels, the variables JS and PVW were scaled based on enneatypes; for example, enneatypes 1 to 3 corresponded to a low level of JS and a maximum score of 66 points on the OJS (Additional file 2). To confirm the agreement between PVW and JS, Cohen’s kappa coefficient was calculated. The data were processed in Stata v15.

The datasets generated and analysed during the current study [26] are available in the [Figshare] repository, [Persistent web link to datasets], DOI [https://doi.org/10.6084/m9.figshare.14308937.v1].

Ethics

The study was approved according to Resolution 292/2018-D-FCEA of Universidad Nacional Agraria de la Selva (Peru). Informed consent was obtained from all study participants, respecting their privacy and free will.

Results

We simultaneously used the OJS and the SPV-Health to evaluate 188 health professionals from 10 health centers in different Peruvian cities. Of these professionals, 94 had experienced PVW, and another 94 had not (Table 1). The samples were homogenous in terms of gender and work activity according to the relevant tests (p < 0.163; p < 0.024), thus fulfilling the conditions indicated above [12]. The average age of the participants was 36.8 ± 10.5 years, and the predominant occupational group was health care providers (80.3% of the total), of whom 18.6% worked in emergency services (Table 1).

Table 1 Social and labour characteristics of the population under study

Sensitivity and specificity

The results showed a statistically significant ROC curve, with an AUC = 0.899 (higher than the suggested minimum of AUC = 0.70) [20, 25] and a standard error (SE) = 0.02, p < 0.01 (95% CI 0.855–0.942), demonstrating that the individuals could be identified randomly using the SPV-Health scale. The scale had a good capacity to classify those who have and those who have not experienced PVW: 89% versus 94% of cases, respectively (Fig. 1). The cut-off point that maximized sensitivity (S = 0.94) and specificity (E = 0.89) was 35 out of a maximum score of 73 on the scale, which is equivalent to good predictive coordinates (ROC curve: ≥ 0.80) [25]. The maximum Youden index (J = 0.83) indicated an adequate predictive limit and discriminative capacity [20].

Fig. 1
figure 1

ROC curve for psychological violence at work evaluated using the SPV-Health scale

Job satisfaction as an external criterion

The magnitude of concordance or reproducibility between two categories of JS and levels of PVW among health professionals was good (k = 0.766), given that k = 0.61–0.80 is adequate (Table 2). When groupings of three categories were performed for PVW and JS, the agreement was also significant according to both the chi-square test and Fisher’s exact test (Additional file 3).

Table 2 Concordance between psychological violence at work and job satisfaction

Although the normal distribution of scalar scores was limited, the degree of nonparametric negative correlation between PVW and JS was high (ρ = − 0.850; p < 0.0001), which confirms that when the PVW levels increase, the perception of JS decreases, and vice versa. Up to 81.7% of the changes in JS were explained by PVW. The JS groups differed in terms of the socio-occupational characteristics of “origin” and “activities” but were similar in terms of “employment status” and “sex” (Additional file 4).

Discussion

This study confirms that the SPV-Health scale shows good randomness for classifying individuals with and without PVW and establishes the cut-off point that maximizes the scale’s sensitivity and specificity (89% and 94%, respectively). We also found that PVW explains up to 81.7% of the changes in the criterion factors of extrinsic and intrinsic JS.

Taking into account the evidence gaps described above, which are related to the usefulness of the instruments for evaluating PVW in the health sector (a specific and contextual area [12]), we highlight three indirectly binding aspects:

  1. a)

    The SPV-Health, which evaluates violence by agents internal to the workplace [1,2,3], differs from the inventory of Diaz et al. [16] in terms of the cut-off scores used to determine high levels of violence and psychological harassment (45 in the latter versus 35 in the SPV-Health); another difference is that the SPV-Health encompasses physical and verbal violence generated by internal and external agents [16]. In theory, these represent hierarchical violence [2, 3] and horizontal violence (Type III) and external violence (Type II) [12].

  2. b)

    The ROC curve results in other areas [3, 19] provide conclusive support for the SPV-Health due to its discriminative capacity (AUC = 0.89 versus the limit of AUC > 0.80) [19] and its randomness of selection, which are adequate in both cases [3, 19].

  3. c)

    Regarding the Pakistani stress questionnaire (AUC = 0.64; kappa = 0.84 [6] versus AUC = 0.89; k = − 0.766 in the present study), the value of the SPV-Health is reinforced in clinical and legal environments, given the convergence of stress as an underlying factor [6]. A decrease in JS negatively affects organizational commitment [5] and increases the levels of stress and exhaustion in health workers [2, 10], and PVW becomes a reason for leaving the job [12].

To the best of our knowledge, this is the first study to evaluate the practical application of and prospects for using the SPV-Health, which uses uniform criteria to classify PVW [3, 4] and makes it possible to collect baseline and follow-up responses within the framework of clinical psychology and organizational psychology [3, 6].

Conclusion

The ROC curve analysis indicates the good randomness of the SPV-Health instrument and establishes that the cut-off point for the scale’s maximum sensitivity and specificity is 35 (out of a maximum score of 73). The scale can be used in different health centres in Peru. The SPV-Health exhibits a good ability to discriminate between individuals with PVW and those without PVW (detecting 89% of those with PVW and 94% of those without PVW).

Limitations

The limitations of this study are related to the low level of quality control in the interviewers’ application of the instrument. The AUC estimation was performed with 94% of the required clinical sample size (n = 188/200) [12]. However, as our comparison groups had the same sample sizes, the coordinates of the determined ROC curve maintained their current positions [12, 22], which increases the usefulness of the SPV-Health scale. The SPV-Health construct validation did not include confirmatory factor analysis given the absence of an additional evaluation sample [22]. Future studies may address the interaction of PWV and resilience and perform stratified analysis.

Availability of data and materials

The dataset used and/or analysed during the study is available at the following link: [Figshare]. They can also be obtained from the authors of this article [26].

Abbreviations

SPV-Health:

Scale for Assessing Psychological Violence in Health care

PVW:

Psychological violence at work

ROC:

Receiver operating characteristic

AUC:

Area under the curve

SE:

Standard error

S:

Sensitivity

E:

Specificity

JS:

Job satisfaction

OJS:

Overall Job Satisfaction Scale

References

  1. Organización Internacional del Trabajo, Consejo Internacional de Enfermeras, Organización Mundial de la Salud. Directrices marco para afrontar la violencia laboral en el sector de la salud. Ginebra: OIT; 2002.

    Google Scholar 

  2. Li P, Xing K, Qiao H, Fang H, Ma H, Jiao M, et al. Psychological violence against general practitioners and nurses in Chinese township hospitals: incidence and implications. Health Qual Life Outcomes. 2018;16:117.

    Article  Google Scholar 

  3. Schindeler E, Reynald DM. What is the evidence? Preventing psychological violence in the workplace. Aggress Violent Behav. 2017;36:25–33.

    Article  Google Scholar 

  4. Hasan MI, Hassan MZ, Bulbul MMI, Joarder T, Chisti MJ. Iceberg of workplace violence in health sector of Bangladesh. BMC Res Notes. 2018;11:702.

    Article  Google Scholar 

  5. Vévoda J, Vévodová Š, Nakládalová M. Psychosocial risks in healthcare. Cas Lek Cesk. 2018;157:411–8.

    PubMed  Google Scholar 

  6. Artani A, Kamal AK, Azam SI, Artani M, Bhamani SS, Saif M, et al. Validation of the Recent Life Changes Questionnaire (RLCQ) for stress measurement among adults residing in urban communities in Pakistan. BMC Psychol. 2019;7:66.

    Article  Google Scholar 

  7. Schablon A, Wendeler D, Kozak A, Nienhaus A, Steinke S. Prevalence and consequences of aggression and violence towards nursing and care staff in Germany—a survey. Int J Environ Res Public Health. 2018;15:1274.

    Article  Google Scholar 

  8. Shea T, Sheehan C, Donohue R, Cooper B, De Cieri H. Occupational violence and aggression experienced by nursing and caring professionals. J Nurs Scholarsh. 2017;49:236–43.

    Article  Google Scholar 

  9. Volz NB, Fringer R, Walters B, Kowalenko T. Prevalence of horizontal violence among emergency attending physicians, residents, and physician assistants. West J Emerg Med. 2017;18:213–8.

    Article  Google Scholar 

  10. Mento C, Silvestri MC, Bruno A, Muscatello MRA, Cedro C, Pandolfo G, et al. Workplace violence against healthcare professionals: a systematic review. Aggress Violent Behav. 2020;51:101381.

    Article  Google Scholar 

  11. Braun-Lewensohn O, Sagy S. Community resilience and sense of coherence as protective factors in explaining stress reactions: comparing cities and rural communities during missiles attacks. Community Ment Health J. 2014;50:229–34.

    Article  Google Scholar 

  12. Nowrouzi-Kia B, Isidro R, Chai E, Usuba K, Chen A. Antecedent factors in different types of workplace violence against nurses: a systematic review. Aggress Violent Behav. 2019;44:1–7.

    Article  Google Scholar 

  13. Travetto C, Daciuk N, Fernández S, Ortiz P, Mastandueno R, Prats M, et al. Agresiones hacia profesionales en el ámbito de la salud. Rev Panam Salud Publica. 2015;38:307–15.

    PubMed  Google Scholar 

  14. Llor-Esteban B, Sánchez-Muñoz M, Ruiz-Hernández JA, Jiménez-Barbero JA. User violence towards nursing professionals in mental health services and emergency units. Eur J Psychol Appl Leg Context. 2017;9:33–40.

    Article  Google Scholar 

  15. Pando M, Aranda C, Preciado L, Franco SA, Salazar JE. Validez y confiabilidad del inventario de violencia y acoso psicológico e-n el trabajo (IVAPT-Pando). Enseñ Invest Psicol. 2015;11:319–32.

    Google Scholar 

  16. Díaz X, Mauro A, Toro JP, Villarroel C, Campos C. Validación del inventario de violencia y acoso psicológico en el trabajo—IVAPT-Pando—en tres ámbitos laborales chilenos. Cienc Trab. 2015;17:7–14.

    Article  Google Scholar 

  17. American Educational Research Association, American Psychological Association (APA), National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.

    Google Scholar 

  18. Coulacoglou C, Saklofske DH. Psychometrics and psychological assessment: principles and applications. Chicago: Elsevier; 2018.

    Google Scholar 

  19. Harris AHS. Three critical questions that should be asked before using prediction models for clinical decision support. JAMA Netw Open. 2019;2:e196661.

    Article  Google Scholar 

  20. Carter JV, Pan J, Rai SN, Galandiuk S. ROC-ing along: evaluation and interpretation of receiver operating characteristic curves. Surgery. 2016;159:1638–45.

    Article  Google Scholar 

  21. Archer L, Snell K, Ensor J, Hudda M, Collins GS, Riley R. Minimum sample size for external validation of a clinical prediction model with a continuous outcome. Stat Med. 2021;40:133–46.

    Article  Google Scholar 

  22. Abregú LF, Galván YJ, Gonzáles CK, Cruz GE. Psychometric validation of the Peruvian version of a short scale to -assess psychological violence in health professionals: a cross-sectional study. Medwave. 2021;21(7): e8452. https://doi.org/10.5867/medwave.2021.07.8452.

    Article  Google Scholar 

  23. Aravena PC, Moraga J, Cartes-Velásquez R, Manterola C. Validez y confiabilidad en investigación odontológica. Int J Odontostomatol. 2014;8:69–75.

    Article  Google Scholar 

  24. Lloret-Segura S, Ferreres-Traver A, Hernández-Baeza A, Tomás-Marco I. Exploratory item factor analysis: a practical guide revised and up-dated. An Psicol. 2014;30:1151–69.

    Article  Google Scholar 

  25. Hajian-Tilaki K. Receiver Operating Characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med. 2013;4:627–35.

    PubMed  PubMed Central  Google Scholar 

  26. Abregú LF, dos Santos R. Data set on variables of psychological violence at work (PVW), job satisfaction (JS) and socio-occupational variables. Health Centres in Peru, 2019–2020. Figshare. 2021. https://doi.org/10.6084/m9.figshare.14308937.v1

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

Study concept and design: LFAT, RDSR. Analysis and interpretation of data: LFAT, RDSR. Drafting of the manuscript: LFAT. Critical revision of the manuscript for important intellectual content: LFAT, RDSR. Statistical analysis: LFAT, RDSR. Obtained funding: RDSR. Administrative, technical, or material support: RDSR. Study supervision: LFAT. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Luis Fidel Abregú Tueros.

Ethics declarations

Ethics approval and consent to participate

The study was approved according to Resolution 292/2018-D-FCEA from Universidad Nacional Agraria de la Selva (Perú). Written informed consent was obtained from the participants with respect to their identity and willingness to participate.

Consent for publication

Not applicable.

Competing interests

The author declares no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: SPV-Health.

A brief Scale to Assess Psychological Violence in Health Professionals that provides socio-occupational information and 22 anonymous responses on psychological violence that occurred during the last 6 months.

Additional file 2: Scales for the division of variables.

The table shows the “enneatypes” (1–9) that were used to divide the “JS” and the “PVW” variables into three levels. For the lowest level, the base scores of the “OJS” scale are located within Enneatypes 1–3. In the second and sixth columns, the ranges and frequency distribution coefficients of the scores for each enneatype are shown.

Additional file 3: Frequency agreement according to “PVW” and “JS” levels.

The table shows the degree of agreement according to three levels (low, medium, high) evaluated by two tests (chi-square and Fisher’s exact test) and the corresponding relative frequencies.

Additional file 4: Job satisfaction according to socio-labour characteristics.

The results for the comparison of “JS” are presented according to the number of cases in each of two groups for “origin”, “employment status” and “sex” and in each of four groups for “activities”.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abregú Tueros, L.F., dos Santos Rosa, R. Sensitivity and specificity of a brief scale to evaluate psychological violence at work in Peruvian health professionals. BMC Res Notes 15, 62 (2022). https://doi.org/10.1186/s13104-022-05959-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-022-05959-8

Keywords

  • Workplace violence
  • Utility of scales
  • ROC curve
  • Cut-off point
  • Health professionals
  • Health care workers