- Research note
- Open Access
Sensitivity and specificity of a brief scale to evaluate psychological violence at work in Peruvian health professionals
BMC Research Notes volume 15, Article number: 62 (2022)
Most studies in Latin America that evaluate psychological violence at work (PVW) focus on measuring occurrences of PVW. However, the discriminative validity and randomness of instruments used for evaluating incidents of PVW that are generated by agents internal to the workplace in the Peruvian health sector have not yet been studied. The objective of this study was to determine the sensitivity and specificity of the Scale of Psychological Violence in Health Professionals (SVP-Health) in the Peruvian population. For this purpose, a cross-sectional study based on the two-stage administration of guided surveys and ROC (receiver operating characteristic) curve analysis was performed.
The study included 188 professionals from ten care centres in Peru. The professionals were divided into two groups of 94 subjects: those who had experienced PVW and those who had not. The average age of the participants was 36.8 ± 10.5 years; their work experience ranged between one and 35 years; and 59% of the sample were women. According to an analysis based on the ROC curve, (a) there was a significant area under the curve (AUC = 0.899) with adequate randomness; and (b) the SPV-Health adequately distinguished subjects with PVW from those without PVW (89% versus 94%).
Psychological violence at work (PVW) refers to an intentional action in which workers are verbally assaulted, threatened and/or humiliated during their professional practice [1, 2] in ways that do not involve physical violence . PVW is exacerbated in the health sector due to the precariousness of working conditions, which commonly involve psychosocial risk factors [4,5,6]. There are no differences in the rates of PVW between developed and underdeveloped countries, between health groups, or between stable and temporary work conditions [7,8,9,10]; however, at the individual level, the effects of PVW can vary according to the individual’s resilience [6, 11].
At the global level, there is a considerable variety of instruments available for evaluating and monitoring the prevalence and impact of PVW [3, 10], but few studies have examined Peruvian health professionals’ experiences with PVW perpetrated by internal agents (that is, superiors or colleagues) [9, 12] and PVW related to interpersonal conflicts or demands. This is partly because some analytical tools, for example, structured formats or the aggressive behaviour scale [10, 13, 14], have been validated for Spain, Mexico, Colombia, Ecuador, Chile and Bolivia but not for the Peru.
Specifically, in Bolivia, Ecuador and Chile, although the convergent validation of the General Health Questionnaire (12 items) has been established and diagnostic scales have been created to measure violence and psychological harassment (including external violence) in workers in the service, industry, commerce and education sectors, the sensitivity and specificity of these diagnostic scales have not been determined [15, 16].
The instruments used in Peru to evaluate PVW in different health care settings have limitations, since their usefulness and the classifications they propose have not been established and, consequently, they do not meet the standards of use for psychological tests [17, 18].
Due to the limitations of the existing measures, it is necessary to develop an instrument with sufficient utility, discriminative capacity and randomness to classify workers exposed to PVW . In addition, it is necessary to standardize the criteria for the interpretation of the results to identify, evaluate and compare the prevalence of PVW according to location (Lima versus other cities), work activity (nursing, medicine and others) and employment status (stable versus temporary) [12, 17] to verify corrective actions and establish baselines in matters of mental health at work. The lack of tools with reference scales and discriminative capacity hinders standardized evaluations of PVW.
This report complements the Scale of Psychological Violence in Health Professionals (SPV-Health) assessment and allows us to answer the following question: What is the best cut-off point for classifying cases with and without psychological violence according to the SPV-Health? Our objective is to determine the sensitivity and specificity of the SPV-Health in the Peruvian population.
This is a cross-sectional study based on the administration of guided surveys in two stages: (a) the classification of groups of equal size comprised of participants with and without PVW according to the external criterion of job satisfaction (JS), which was evaluated using an overall job satisfaction scale (OJS); and b) evaluation of the sensitivity and specificity of the SPV-Health in both groups (participants with and without PVW). The surveys were conducted between December 2019 and February 2020. To determine the sample size, the minimum recommendations established for external validation and receiver operating characteristic (ROC) curves were adopted: groups of n ≥ 176–200 participants [20, 21] with similar sex distributions . No calculation was required.
The inclusion criteria for the participants were (a) complete responses to both assessment instruments (the OJS and SPV-Health) and (b) having worked at least 6 months before the time of the evaluation. The exclusion criterion was less than 2 years of work experience .
SPV-Health (short scale used to evaluate psychological violence in health professionals)
For this study, a scale was validated that comprises 22 items referring to three types of violence in the context of Peruvian health care settings: isolation, intimidation and discreditation . Each item is rated on a 4-point Likert scale (ranging from 1, “never” to 4, “always”) (see Additional file 1). The scale showed good reliability and validity in a population of 316 health professionals . Its properties are as follows:
Good content validity, evaluated with the judgement of five experts (doctors, nurses and teaching psychologists), with a Kendall coefficient (W = 0.509; p < 0.026) that exceeded the established minimum values .
Appropriate construct validity, with factors that explain up to 54.3% of the total variance, which exceeds the minimum (50%) ; confirmatory factors of moderate adjustment; root mean square error of approximation and standardized residual of the mean square root above the acceptability limit; and acceptable comparative fit and Tucker–Lewis indices (0.60).
Good convergent validity with the OJS (r = − 0.769; p < 0.0003).
Good internal consistency reliability, with a global Cronbach’s alpha coefficient of α = 0.803 and an item Cronbach’s alpha of α = 0.800 (acceptable values are α = 0.70–0.80) . Reliability was assessed in terms of response stability; there were significant differences between the first and second applications, which were separated by a 4-month interval (p < 0.008).
Overall Job Satisfaction Scale (OJS)
The OJS consists of 15 items grouped into two subscales: (a) intrinsic job satisfaction, which is related to task-based factors such as recognition, responsibility and promotion (7 items); and (b) extrinsic satisfaction, which is related to satisfaction with the organization, schedules, wages and physical conditions of the work (8 items). Each item is rated on a 7-point Likert scale (1, “very dissatisfied” to 7, “very satisfied”). The scale presented good reliability and validity in 518 Spanish nurses , with an overall reliability of alpha 0.75. Its construct reflects experiences and emotional responses at work .
The study subjects were informed about the objectives of the research, and their voluntary participation was recorded through a letter of consent in which the participants’ anonymity and the confidentiality of the data were guaranteed. The simultaneous collection of data was carried out by organizational psychologists and health professionals.
To evaluate the sensitivity and specificity of the SPV-Health in groups of participants with and without PVW, a ROC curve analysis was performed, and the area under the curve (AUC), the standard error (SE) and the cut-off point that maximizes the sensitivity–specificity of the instrument were determined [20, 25]. To divide the results into three levels, the variables JS and PVW were scaled based on enneatypes; for example, enneatypes 1 to 3 corresponded to a low level of JS and a maximum score of 66 points on the OJS (Additional file 2). To confirm the agreement between PVW and JS, Cohen’s kappa coefficient was calculated. The data were processed in Stata v15.
The study was approved according to Resolution 292/2018-D-FCEA of Universidad Nacional Agraria de la Selva (Peru). Informed consent was obtained from all study participants, respecting their privacy and free will.
We simultaneously used the OJS and the SPV-Health to evaluate 188 health professionals from 10 health centers in different Peruvian cities. Of these professionals, 94 had experienced PVW, and another 94 had not (Table 1). The samples were homogenous in terms of gender and work activity according to the relevant tests (p < 0.163; p < 0.024), thus fulfilling the conditions indicated above . The average age of the participants was 36.8 ± 10.5 years, and the predominant occupational group was health care providers (80.3% of the total), of whom 18.6% worked in emergency services (Table 1).
Sensitivity and specificity
The results showed a statistically significant ROC curve, with an AUC = 0.899 (higher than the suggested minimum of AUC = 0.70) [20, 25] and a standard error (SE) = 0.02, p < 0.01 (95% CI 0.855–0.942), demonstrating that the individuals could be identified randomly using the SPV-Health scale. The scale had a good capacity to classify those who have and those who have not experienced PVW: 89% versus 94% of cases, respectively (Fig. 1). The cut-off point that maximized sensitivity (S = 0.94) and specificity (E = 0.89) was 35 out of a maximum score of 73 on the scale, which is equivalent to good predictive coordinates (ROC curve: ≥ 0.80) . The maximum Youden index (J = 0.83) indicated an adequate predictive limit and discriminative capacity .
Job satisfaction as an external criterion
The magnitude of concordance or reproducibility between two categories of JS and levels of PVW among health professionals was good (k = 0.766), given that k = 0.61–0.80 is adequate (Table 2). When groupings of three categories were performed for PVW and JS, the agreement was also significant according to both the chi-square test and Fisher’s exact test (Additional file 3).
Although the normal distribution of scalar scores was limited, the degree of nonparametric negative correlation between PVW and JS was high (ρ = − 0.850; p < 0.0001), which confirms that when the PVW levels increase, the perception of JS decreases, and vice versa. Up to 81.7% of the changes in JS were explained by PVW. The JS groups differed in terms of the socio-occupational characteristics of “origin” and “activities” but were similar in terms of “employment status” and “sex” (Additional file 4).
This study confirms that the SPV-Health scale shows good randomness for classifying individuals with and without PVW and establishes the cut-off point that maximizes the scale’s sensitivity and specificity (89% and 94%, respectively). We also found that PVW explains up to 81.7% of the changes in the criterion factors of extrinsic and intrinsic JS.
Taking into account the evidence gaps described above, which are related to the usefulness of the instruments for evaluating PVW in the health sector (a specific and contextual area ), we highlight three indirectly binding aspects:
The SPV-Health, which evaluates violence by agents internal to the workplace [1,2,3], differs from the inventory of Diaz et al.  in terms of the cut-off scores used to determine high levels of violence and psychological harassment (45 in the latter versus 35 in the SPV-Health); another difference is that the SPV-Health encompasses physical and verbal violence generated by internal and external agents . In theory, these represent hierarchical violence [2, 3] and horizontal violence (Type III) and external violence (Type II) .
The ROC curve results in other areas [3, 19] provide conclusive support for the SPV-Health due to its discriminative capacity (AUC = 0.89 versus the limit of AUC > 0.80)  and its randomness of selection, which are adequate in both cases [3, 19].
Regarding the Pakistani stress questionnaire (AUC = 0.64; kappa = 0.84  versus AUC = 0.89; k = − 0.766 in the present study), the value of the SPV-Health is reinforced in clinical and legal environments, given the convergence of stress as an underlying factor . A decrease in JS negatively affects organizational commitment  and increases the levels of stress and exhaustion in health workers [2, 10], and PVW becomes a reason for leaving the job .
To the best of our knowledge, this is the first study to evaluate the practical application of and prospects for using the SPV-Health, which uses uniform criteria to classify PVW [3, 4] and makes it possible to collect baseline and follow-up responses within the framework of clinical psychology and organizational psychology [3, 6].
The ROC curve analysis indicates the good randomness of the SPV-Health instrument and establishes that the cut-off point for the scale’s maximum sensitivity and specificity is 35 (out of a maximum score of 73). The scale can be used in different health centres in Peru. The SPV-Health exhibits a good ability to discriminate between individuals with PVW and those without PVW (detecting 89% of those with PVW and 94% of those without PVW).
The limitations of this study are related to the low level of quality control in the interviewers’ application of the instrument. The AUC estimation was performed with 94% of the required clinical sample size (n = 188/200) . However, as our comparison groups had the same sample sizes, the coordinates of the determined ROC curve maintained their current positions [12, 22], which increases the usefulness of the SPV-Health scale. The SPV-Health construct validation did not include confirmatory factor analysis given the absence of an additional evaluation sample . Future studies may address the interaction of PWV and resilience and perform stratified analysis.
Availability of data and materials
The dataset used and/or analysed during the study is available at the following link: [Figshare]. They can also be obtained from the authors of this article .
Scale for Assessing Psychological Violence in Health care
Psychological violence at work
Receiver operating characteristic
Area under the curve
Overall Job Satisfaction Scale
Organización Internacional del Trabajo, Consejo Internacional de Enfermeras, Organización Mundial de la Salud. Directrices marco para afrontar la violencia laboral en el sector de la salud. Ginebra: OIT; 2002.
Li P, Xing K, Qiao H, Fang H, Ma H, Jiao M, et al. Psychological violence against general practitioners and nurses in Chinese township hospitals: incidence and implications. Health Qual Life Outcomes. 2018;16:117.
Schindeler E, Reynald DM. What is the evidence? Preventing psychological violence in the workplace. Aggress Violent Behav. 2017;36:25–33.
Hasan MI, Hassan MZ, Bulbul MMI, Joarder T, Chisti MJ. Iceberg of workplace violence in health sector of Bangladesh. BMC Res Notes. 2018;11:702.
Vévoda J, Vévodová Š, Nakládalová M. Psychosocial risks in healthcare. Cas Lek Cesk. 2018;157:411–8.
Artani A, Kamal AK, Azam SI, Artani M, Bhamani SS, Saif M, et al. Validation of the Recent Life Changes Questionnaire (RLCQ) for stress measurement among adults residing in urban communities in Pakistan. BMC Psychol. 2019;7:66.
Schablon A, Wendeler D, Kozak A, Nienhaus A, Steinke S. Prevalence and consequences of aggression and violence towards nursing and care staff in Germany—a survey. Int J Environ Res Public Health. 2018;15:1274.
Shea T, Sheehan C, Donohue R, Cooper B, De Cieri H. Occupational violence and aggression experienced by nursing and caring professionals. J Nurs Scholarsh. 2017;49:236–43.
Volz NB, Fringer R, Walters B, Kowalenko T. Prevalence of horizontal violence among emergency attending physicians, residents, and physician assistants. West J Emerg Med. 2017;18:213–8.
Mento C, Silvestri MC, Bruno A, Muscatello MRA, Cedro C, Pandolfo G, et al. Workplace violence against healthcare professionals: a systematic review. Aggress Violent Behav. 2020;51:101381.
Braun-Lewensohn O, Sagy S. Community resilience and sense of coherence as protective factors in explaining stress reactions: comparing cities and rural communities during missiles attacks. Community Ment Health J. 2014;50:229–34.
Nowrouzi-Kia B, Isidro R, Chai E, Usuba K, Chen A. Antecedent factors in different types of workplace violence against nurses: a systematic review. Aggress Violent Behav. 2019;44:1–7.
Travetto C, Daciuk N, Fernández S, Ortiz P, Mastandueno R, Prats M, et al. Agresiones hacia profesionales en el ámbito de la salud. Rev Panam Salud Publica. 2015;38:307–15.
Llor-Esteban B, Sánchez-Muñoz M, Ruiz-Hernández JA, Jiménez-Barbero JA. User violence towards nursing professionals in mental health services and emergency units. Eur J Psychol Appl Leg Context. 2017;9:33–40.
Pando M, Aranda C, Preciado L, Franco SA, Salazar JE. Validez y confiabilidad del inventario de violencia y acoso psicológico e-n el trabajo (IVAPT-Pando). Enseñ Invest Psicol. 2015;11:319–32.
Díaz X, Mauro A, Toro JP, Villarroel C, Campos C. Validación del inventario de violencia y acoso psicológico en el trabajo—IVAPT-Pando—en tres ámbitos laborales chilenos. Cienc Trab. 2015;17:7–14.
American Educational Research Association, American Psychological Association (APA), National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
Coulacoglou C, Saklofske DH. Psychometrics and psychological assessment: principles and applications. Chicago: Elsevier; 2018.
Harris AHS. Three critical questions that should be asked before using prediction models for clinical decision support. JAMA Netw Open. 2019;2:e196661.
Carter JV, Pan J, Rai SN, Galandiuk S. ROC-ing along: evaluation and interpretation of receiver operating characteristic curves. Surgery. 2016;159:1638–45.
Archer L, Snell K, Ensor J, Hudda M, Collins GS, Riley R. Minimum sample size for external validation of a clinical prediction model with a continuous outcome. Stat Med. 2021;40:133–46.
Abregú LF, Galván YJ, Gonzáles CK, Cruz GE. Psychometric validation of the Peruvian version of a short scale to -assess psychological violence in health professionals: a cross-sectional study. Medwave. 2021;21(7): e8452. https://doi.org/10.5867/medwave.2021.07.8452.
Aravena PC, Moraga J, Cartes-Velásquez R, Manterola C. Validez y confiabilidad en investigación odontológica. Int J Odontostomatol. 2014;8:69–75.
Lloret-Segura S, Ferreres-Traver A, Hernández-Baeza A, Tomás-Marco I. Exploratory item factor analysis: a practical guide revised and up-dated. An Psicol. 2014;30:1151–69.
Hajian-Tilaki K. Receiver Operating Characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med. 2013;4:627–35.
Abregú LF, dos Santos R. Data set on variables of psychological violence at work (PVW), job satisfaction (JS) and socio-occupational variables. Health Centres in Peru, 2019–2020. Figshare. 2021. https://doi.org/10.6084/m9.figshare.14308937.v1
Ethics approval and consent to participate
The study was approved according to Resolution 292/2018-D-FCEA from Universidad Nacional Agraria de la Selva (Perú). Written informed consent was obtained from the participants with respect to their identity and willingness to participate.
Consent for publication
The author declares no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A brief Scale to Assess Psychological Violence in Health Professionals that provides socio-occupational information and 22 anonymous responses on psychological violence that occurred during the last 6 months.
The table shows the “enneatypes” (1–9) that were used to divide the “JS” and the “PVW” variables into three levels. For the lowest level, the base scores of the “OJS” scale are located within Enneatypes 1–3. In the second and sixth columns, the ranges and frequency distribution coefficients of the scores for each enneatype are shown.
The table shows the degree of agreement according to three levels (low, medium, high) evaluated by two tests (chi-square and Fisher’s exact test) and the corresponding relative frequencies.
The results for the comparison of “JS” are presented according to the number of cases in each of two groups for “origin”, “employment status” and “sex” and in each of four groups for “activities”.
About this article
Cite this article
Abregú Tueros, L.F., dos Santos Rosa, R. Sensitivity and specificity of a brief scale to evaluate psychological violence at work in Peruvian health professionals. BMC Res Notes 15, 62 (2022). https://doi.org/10.1186/s13104-022-05959-8
- Workplace violence
- Utility of scales
- ROC curve
- Cut-off point
- Health professionals
- Health care workers