Skip to main content

Advertisement

You are viewing the new article page. Let us know what you think. Return to old version

Validity and reliability of the Utrecht Work Engagement Scale-Student Version in Sri Lanka

BMC Research Notesvolume 11, Article number: 277 (2018) | Download Citation

Abstract

Objective

The present study was aimed at assessing the validity and the reliability of the Sinhala version of the Utrecht Work Engagement Scale-Student Version (UWES-S) among collegiate cycle students in Sri Lanka.

Results

The 17-item UWES-S was translated to Sinhala and the judgmental validity was assessed by a multi-disciplinary panel of experts. Construct validity of the UWES-S was appraised by using multi-trait scaling analysis and exploratory factor analysis (EFA) on data obtained from a sample of 194 grade thirteen students in the Kurunegala district, Sri Lanka. Reliability of the UWES-S was assessed by using internal consistency and test–retest reliability. Except for item 13, all other items showed good psychometric properties in judgemental validity, item-convergent validity and item-discriminant validity. EFA using principal component analysis with Oblimin rotation, suggested a three-factor solution (including vigor, dedication and absorption subscales) explaining 65.4% of the total variance for the 16-item UWES-S (with item 13 deleted). All three subscales show high internal consistency with Cronbach’s α coefficient values of 0.867, 0.819, and 0.903 and test–retest reliability was high (p < 0.001). Hence, the Sinhala version of the 16-item UWES-S is a valid and a reliable instrument to assess work engagement among collegiate cycle students in Sri Lanka.

Introduction

Keeping on par with the emerging trend towards a positive psychology focusing on optimal functioning rather than on malfunctioning, a growing enthusiasm is evident in student engagement research during the last few decades [1].

Work engagement is defined as, “a positive, fulfilling, work-related state of mind that is characterised by vigor (VI), dedication (DE), and absorption (AB). Rather than a momentary and specific state, engagement refers to a more persistent and pervasive affective-cognitive state that is not focused on any particular object, event, individual or behaviour” [2].

Engagement of students in their academic environment has been studied in numerous disciplines [3,4,5,6,7,8,9,10] and the global literature suggests that students’ engagement is positively linked with their academic performances [11, 12] and positive academic outcomes [13,14,15,16]. Hence, educators and policymakers are increasingly focusing on student engagement as a means of addressing problems of negative academic outcomes of varying student groups [17].

Amongst the different assessment tools of student work engagement, the most commonly used instrument is the Utrecht Work Engagement Scale-Student Version (UWES-S) [2, 18,19,20,21,22], which is a 17-item self-report measure assessing VI, DE, and AB subscales.

Even though a plethora of research has been conducted on work engagement among varying student populations across the globe, there is a paucity of literature in the South Asian context with no published literature on work engagement among Sri Lankan students, mainly owing to the lack of validated assessment tools. In the context of ever-increasing burden of mental health problems in the Sri Lankan collegiate cycle students [23,24,25,26], exploration of effective strategies for mental health and wellbeing promotion has become a timely need. Hence, the present study was designed to assess the validity and the reliability of a culturally adapted Sinhala version of the UWES-S among collegiate cycle students in Sri Lanka.

Main text

Methods

Structure of the UWES-S

The 17-item UWES-S is a self-administered questionnaire (SAQ) with six, five, and six items assessing VI, DE and AB subscales respectively, measured on a seven-point Likert scale anchored by the response options from 0 (never) to 6 (every day).

Translation and pre-testing of the UWES-S

The 17-item UWES-S was translated to Sinhala by using the forward–backward translation method [27,28,29], involving two independent bilingual translators, who are fluent in Sinhala and English.

The synthesised forward translation of UWES-S was pre-tested among a sample of 25 grade thirteen students outside the study setting. In a subsequent structured interview, the clarity in understanding, acceptability, and comprehension of items and feasibility of using the questionnaire were assessed. None of the items of the questionnaire was claimed to be difficult to understand.

Appraising the judgemental validity of the UWES-S

The face, content, and consensual validity were assessed. Using a modified Delphi technique, a multi-disciplinary panel of experts in the fields of psychiatry, psychology, public health, teaching, student counseling, and medical education has assessed each item on its relevance, appropriateness, and acceptability in the local context for assessing burnout among collegiate cycle students based on a rating scale from 0 (strong disagreement) to 10 (strong agreement). At the end of this iterative process, except for the item 13 stating, “When studying, I am very resilient, mentally”, all other items had a median score more than 7 for all the aspects. Thus, it was decided to include all 17 items for the assessment of construct validity.

Study design and setting

This school-based, cross-sectional validation study was conducted in the Kurunegala district, North Western province, Sri Lanka from May 2014 to April 2015. In Sri Lanka, the collegiate cycle in the education system consists of grade twelve and thirteen. Three Sinhala medium government schools in the Kurunegala district having all four collegiate cycle subject streams, viz., Science, Arts, Commerce and Technology were selected.

Study participants

From the selected schools, three grade thirteen classes each were selected representing both male and female students studying in all four subject streams. A total of 194 students participated in the study in their classrooms and each participant filled the SAQ independently. The response rate was 100.0% and 55.2% (n = 107) of the sample were females. The mean age was 18.3 years (SD = 0.43 years). The majority of students were studying in the Science stream (n = 78, 40.2%). The numbers of students in the Arts, Commerce and the Technology streams were 60 (30.9%), 41 (21.2%), and 15 (7.7%) respectively.

Data analysis

Data were analysed by using the Statistical Package for Social Sciences (SPSS) version 17.0. Preparatory data analysis showed that there were no violations of the assumptions related to the data analytic techniques. Given that the participants to variables ratio was 11.4, the sample size was adequate for factor analysis [30]. Even though there were few items that showed non-normal distribution of data, it is considered to be a ubiquitous phenomenon in psychological assessment research.

Multi-trait scaling analysis

Item-scale correlations were analysed and item-convergent and item-discriminant validity were assessed. A stringent criterion of correlation of 0.40 or greater between an item and its own subscale was considered as a success for assessing item-convergent validity. Furthermore, in assessing item-discriminant validity, items that correlated significantly higher (more than 1.96 standard errors) with its own subscale than with the other two subscales, were considered as scaling successes.

Exploratory factor analysis (EFA)

EFA was conducted by Principal Component Analysis (PCA) with Oblimin rotation. Kaiser’s criterion/eigenvalue, scree plot and parallel analysis were used to decide the number of factors to retain. The factors that lead to a meaningful interpretation and theoretical sense were ultimately selected.

Assessment of reliability

Reliability was assessed using internal consistency and test–retest reliability by re-administering the SAQ to a sub sample of 22 grade thirteen students after 2 weeks.

Results

Descriptive statistics of the UWES-S scores

Descriptive statistics of UWES-S subscales are given in Table 1. The mean item score was highest in the AB subscale (4.52, SD = 1.40) and was lowest in the DE subscale (3.70, SD = 1.23).

Table 1 Descriptive statistics of the UWES-S subscale scores among grade thirteen students (n = 194)
Full size table

Multi-trait scaling analysis

Table 2 summarises the results of the multi-trait scaling analysis and as per the predetermined cut-off values, except for item 13, item-convergent validity and item-discriminant validity were confirmed for other 16 items in the UWES-S.

Table 2 Analysis of item-scale correlations to determine item-discriminant scaling successes for the UWES-S scores (n = 194)
Full size table

Exploratory factor analysis

Item 13 was found to have poor psychometric properties in judgemental validity, item-convergent validity, and item-discriminant validity. Deletion of item 13 from the subscale also improved the internal consistency. Hence, both 17-item UWES-S and 16 items of the UWES-S (item 13 deleted) were subjected to PCA. The Kaiser–Meyer–Olkin Measure were 0.851 and 0.855 respectively and Bartlett’s test of sphericity reached statistical significance (p < 0.001) for both versions.

Only three factors had eigenvalues more than 1.0, which cumulatively explained 63.9% and 65.5% of the total variance for the 17-item and 16-item UWES-S versions respectively. For both versions, the parallel analysis showed only two factors with eigenvalues exceeding the corresponding criterion values and the screeplot showed a clear break after the second factor.

However, for both versions, the component matrix in PCA with unrotated loadings revealed a number of items of UWES-S loading on the third factor with values greater than 0.3. Furthermore, the pattern matrix generated in PCA using direct Oblimin rotation revealed that seven items had loading values more than 0.3. Based on this collective evidence, it was decided to ‘force’ a three-factor solution for further investigation.

Even though both Varimax rotation and Oblimin rotation was used, it was decided to report statistics related to Oblimin rotation as the factors were strongly correlated (> 0.3). The rotated three-factor solution revealed a simple structure with all three factors showing a number of strong loadings. In the 17-item UWES-S version, the three-factor solution explained a total of 63.9% of the variance, with factor one contributing 41.4%, factor two contributing 14.9%, and factor three contributing 7.6%, whereas, the corresponding values for the 16-item UWES-S version were 65.4, 44.0, 14.2, and 7.2%. Table 3 shows the pattern and structure matrix for PCA with Oblimin rotation of three-factor solution of 16-item UWES-S.

Table 3 Pattern and structure matrix for PCA with Oblimin rotation of three-factor solution of 16-item UWES-S
Full size table

The interpretation of three-factor solution of the UWES-S with item 13 deleted, was consistent with previous research on the UWES-S, with subscale items of VI, DE, and AB loading strongly on three different factors. Hence, the three-factor solution with item 13 deleted was considered as the validated Sinhala version of the UWES-S.

Assessment of reliability

The impact of each item on the related subscale was assessed by computing Cronbach’s α when the respective item is deleted. None of the items in DE subscale and AB subscale improved the overall Cronbach’s α value; however, deletion of item 13 improved the Cronbach’s α value from 0.696 to 0.867. Hence, item 13 was removed from the UWES-S and the reliability was re-assessed for internal consistency and all three subscales show high internal consistency with Cronbach’s α coefficient values of 0.867, 0.819, and 0.903 for VI, DE, and AB subscales respectively.

There were strong, positive statistically significant (p < 0.001) correlations for each of the three subscales of the UWES-S in test–retest reliability assessment. The correlation coefficients were 0.831, 0.866 and 0.839 for the VI, DE, and AB subscales respectively.

Discussion

The present study was designed to validate the Sinhala version of the UWES-S among Sri Lankan collegiate cycle students addressing an important research vacuum on work engagement research in high school students in the South Asian context. A cross-sectional study design and the sample size of the study deemed appropriate with the study objective.

In assessing the judgemental validity, item 13 (“When studying, I am very resilient, mentally”) had the least median rating score and the main concern raised regarding the item was the difficulty in explaining the phrase “mentally resilience” in Sinhala language. The global literature suggests that modified versions of the UWES-S with removal of several items have been used in work engagement among high school, college, and university students [21, 22, 31].

Even though the three-factor structure of the Utrecht Work Engagement Scale (UWES) has been confirmed in a large number of studies [32,33,34,35,36], the limited number of studies that explored the factor validity of the UWES-S have failed to provide convincing evidence to support the three-factor structure of the UWES-S [37]. Furthermore, the findings of the study conducted by Sonnentag [38], raises concerns about the three-factor structure of UWES. This evidence coupled with the fact that UWES-S is a relatively novel measure in the South Asian context, EFA was employed to assess the construct validity, instead of using confirmatory factor analysis [39].

The three-factor solution of the original 17-item UWES-S, explained 63.9% of the variance, whereas the three-factor solution of the UWES-S with item 13 deleted, explained 65.4% of the variance. Furthermore, the interpretation of the three-factor solution of the UWES-S with item 13 deleted, was consistent with previous research on the UWES-S with subscale items of VI, DE, and AB loading strongly on three different factors [2, 22].

In assessing the internal consistency, deletion of item 13 improved the Cronbach’s α value from 0.696 to 0.867 and all three subscales showed high internal consistency. This finding is consistent with findings of studies using the Portuguese version [22], Spanish version [22], Dutch version [2, 22], Turkish version [21] and Romanian version [20] of the UWES-S.

The present study also revealed that there were statistically significant strong positive correlations for each of the three subscales of UWES-S in the test–retest reliability assessment. Though these findings are consistent with the previous study findings [21], the values were not as high as those reported in that study.

In conclusion, the study findings indicate that the Sinhala version of the 16-item UWES-S is a valid and a reliable instrument to assess work engagement among collegiate cycle grade thirteen students in Sri Lanka. Due to its brevity, ease of administration and sound psychometric properties, it could be used as an effective screening tool at the school level. The study findings pave the way to establish the initial evidence base for the relevance and the applicability of the concept of work engagement in the South Asian context.

Limitations

This study has some limitations. Given that the study sample was recruited not using a probability sampling technique in a selected district of Sri Lanka, the generalisability of the study findings to other populations should be done with caution, considering the variations in educational and cultural contexts. Furthermore, given that the three-factor structure of the UWES-S is established in the present study, confirmatory factor analysis procedures are recommended for future studies.

Abbreviations

AB:

absorption

DE:

dedication

EFA:

exploratory factor analysis

PCA:

principal component analysis

SPSS:

Statistical Package for Social Sciences

UWES-S:

Utrecht Work Engagement Scale-Student Version

VI:

vigor

References

  1. 1.

    Seligman MEP, Csikszentmihalyi M. Positive psychology. Am Psychol. 2000;55(1):5–14. https://doi.org/10.1002/jclp.20593.

  2. 2.

    Schaufeli WB, Bakker AB. Utrecht work engagement scale: preliminary manual. 2003. http://www.wilmarschaufeli.nl/publications/Schaufeli/Test%20Manuals/Test_manual_UWES_English.pdf. Accessed 13 Jan 2015.

  3. 3.

    Cadime I, Lima S, Marques Pinto A, Ribeiro I. Measurement invariance of the Utrecht Work Engagement Scale for Students: a study across secondary school pupils and university students. Eur J Dev Psychol. 2016;13(2):254–63.

  4. 4.

    Casuso-Holgado MJ, Cuesta-Vargas AI, Moreno-Morales N, Labajos-Manzanares MT, Barón-López FJ, Vega-Cuesta M. The association between academic engagement and achievement in health sciences students. BMC Med Educ. 2013;13(1):33.

  5. 5.

    Mostert K, Pienaar J, Gauché C, Jackson LTB. Burnout and engagement in university students: a psychometric analysis of the MBI-SS and UWES-S. S Afr J High Educ. 2007;21(1):147–62.

  6. 6.

    Pekrun R, Linnenbrink EA. Academic emotions and student engagement. Handbook of research on student engagement. Boston: Springer; 2012. p. 259–82.

  7. 7.

    Strauser DR, O’Sullivan D, Wong AWK. Work personality, work engagement, and academic effort in a group of college students. J Employ Couns. 2012;49(2):50–61.

  8. 8.

    Stoliker BE, Lafreniere KD. The influence of perceived stress, loneliness, and learning burnout on University Students’ Educational experience. Coll Stud J. 2015;49(1):146–60.

  9. 9.

    Trowler P, Trowler V. Student engagement evidence summary. York: The Higher Education Academy; 2010. p. 1–15.

  10. 10.

    Zhang Y, Gan Y, Cham H. Perfectionism, academic burnout and engagement among Chinese college students: a structural equation modeling analysis. Personal Individ Differ. 2007;43(6):1529–40.

  11. 11.

    Carini RM, Kuh GD, Klein SP. Student engagement and student learning: testing the linkages. Res High Educ. 2006;47(1):1–32.

  12. 12.

    Pike GR, Kuh GD, Massa-McKinley RC. First-year students’ employment, engagement, and academic achievement: untangling the relationship between work and grades. J Stud Aff Res Pract. 2009;45:560–83.

  13. 13.

    Ouweneel E, Schaufeli WB, Le Blanc PM. Believe, and you will achieve: changes over time in self-efficacy, engagement, and performance. Appl Psychol Heal Well-Being. 2013;5(2):225–47.

  14. 14.

    Salmela-Aro K, Upadyaya K. School burnout and engagement in the context of demands-resources model. Br J Educ Psychol. 2014;84(1):137–51.

  15. 15.

    Siu OL, Bakker AB, Jiang X. Psychological capital among University students: relationships with study engagement and intrinsic motivation. J Happiness Stud. 2014;15(4):979–94.

  16. 16.

    Stoeber J, Childs JH, Hayward JA, Feast AR. Passion and motivation for studying: predicting academic engagement and burnout in university students. Educ Psychol. 2011;31(4):513–28.

  17. 17.

    Fredricks JA, Blumenfeld PC, Paris AH. School engagement: potential of the concept, state of the evidence. Rev Educ Res. 2004;74(1):59–109.

  18. 18.

    Çapri B, Gündüz B, Akbay SE. Utrecht work engagement scale-student forms’ (UWES-SF) adaptation to Turkish, validity and reliability studies, and the mediator role of work engagement between academic procrastination and academic responsibility. Kuram ve Uygulamada Egit Bilim. 2017;17(2):411–35.

  19. 19.

    Gómez HP, Pérez VC, Parra PP, et al. Academic achievement, engagement and burnout among first year medical students. Rev Med Chile. 2015;143(7):930–7.

  20. 20.

    Cazan A-M. Learning motivation, engagement and burnout among University students. Procedia Soc Behav Sci. 2015;187:413–7.

  21. 21.

    Bilge F, Tuzgöl Dost M, Çetin B. Factors affecting burnout and school engagement among high school students: study habits, self-efficacy beliefs, and academic success. Educ Sci Theory Pract. 2014;14(5):1721–7.

  22. 22.

    Schaufeli WB, Martinez IM, Pinto AM, Salanova M, Bakker AB. Burnout and engagement in University students: a cross-national study. J Cross Cult Psychol. 2002;33(5):464–81.

  23. 23.

    UNICEF Sri Lanka. National survey on emerging issues among adolescents in Sri Lanka. 2004. https://www.unicef.org/srilanka/Full_Report.pdf. Accessed 12 Jan 2016.

  24. 24.

    Perera H. Mental health of adolescent school children in Sri Lanka—a national survey. Sri Lanka J Child Health. 2004;33:78–81.

  25. 25.

    Rodrigo C, Welgama S, Gurusinghe J, Wijeratne T, Jayananda G, Rajapakse S. Symptoms of anxiety and depression in adolescent students; a perspective from Sri Lanka. Child Adolesc Psychiatry Ment Health. 2010;4:10.

  26. 26.

    Wickramasinghe ND, Dissanayake DS, Abeywardena GS. Prevalence and correlates of burnout among collegiate cycle students in Sri Lanka: a school-based cross-sectional study. 2017 (Manuscript submitted for publication).

  27. 27.

    Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol. 2010;10(1):13.

  28. 28.

    Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–32.

  29. 29.

    World Health Organization. Process of translation and adaptation of instruments. http://www.who.int/substance_abuse/research_tools/translation/en/. Accessed 11 Feb 2017.

  30. 30.

    Gaskin CJ, Happell B. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud. 2014;51(3):511–21.

  31. 31.

    Tsubakita T, Shimazaki K, Ito H, Kawazoe N. Item response theory analysis of the Utrecht Work Engagement Scale for Students (UWES-S) using a sample of Japanese university and college students majoring medical science, nursing, and natural science. BMC Res Notes. 2017;10:528.

  32. 32.

    Schaufeli WB, Taris TW, Van Rhenen W. Workaholism, burnout, and work engagement: three of a kind or three different kinds of employee well-being? Appl Psychol. 2008;57(2):173–203.

  33. 33.

    Seppälä P, Mauno S, Feldt T, et al. The construct validity of the Utrecht Work Engagement Scale: multisample and longitudinal evidence. J Happiness Stud. 2009;10(4):459–81.

  34. 34.

    Ugwu FO. Work Engagement in Nigeria: adaptation of the Utrecht Work Engagement Scale for Nigerian Samples. Int J Multidiscip Acad Res. 2013;1(3):16–26.

  35. 35.

    Gilchrist RM, Villalobos CEP, Fernández LR. Factorial structure and internal consistency of the Utrecht Work Engagement Scale (Uwes) 17 among health workers of Chile. Liberabit. 2013;19(2):163–71.

  36. 36.

    Shimazu A, Schaufeli WB, Kosugi S, et al. Work engagement in Japan: validation of the Japanese version of the Utrecht Work Engagement Scale. Appl Psychol. 2008;57(3):510–23.

  37. 37.

    Wefald A, Downey R. Construct dimensionality of engagement and its relation with satisfaction. J Psychol Interdiscip Appl. 2009;143(1):91–111.

  38. 38.

    Sonnentag S. Recovery, work engagement, and proactive behavior: a new look at the interface between nonwork and work. J Appl Psychol. 2003;88(3):518–28.

  39. 39.

    Hurley AE, Scandura TA, Schriesheim CA, et al. Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives. J Organ Behav. 1997;18(6):667–83.

Download references

Authors’ contributions

NDW, DSD and GSA were involved in the conception and design of the study. NDW collected, analysed and interpreted data. DSD and GSA made substantial contribution to data analysis and interpretation. NDW prepared the manuscript. DSD and GSA made substantial contribution to revise the manuscript. All authors read and approved the final manuscript.

Acknowledgements

Authors would like to acknowledge all the students who participated in the study for their support, all experts involved in assessing the judgmental validity of the study instrument for their valuable contribution and guidance, and Mrs Shanthi Attanayake and Mrs Bhagya Senanayake for their support during data collection.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The datasets used and analysed during the present study are available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Ethical clearance to conduct the study was obtained from the Ethics Review Committee of the Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka (Reference No: ERC/2014/057). Informed written consent form all the participants were obtained prior to data collection. (All the participants were above the age of 16 years.)

Funding

This work was supported by the University Grants Commission-Sri Lanka, under the Postgraduate Research Grant scheme [Grant Number: UGC/DRIC/PG/2015(I)/RUSL/01]. The funding body did not involve in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Affiliations

  1. Department of Community Medicine, Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka, Saliyapura, 50008, Sri Lanka

    • Nuwan Darshana Wickramasinghe

  2. Department of Community Medicine, Faculty of Medicine, University of Peradeniya, Peradeniya, 20400, Sri Lanka

    • Devani Sakunthala Dissanayake

  3. Teaching Hospital-Kandy, Kandy, 20000, Sri Lanka

    • Gihan Sajiwa Abeywardena

Authors

  1. Search for Nuwan Darshana Wickramasinghe in:

  2. Search for Devani Sakunthala Dissanayake in:

  3. Search for Gihan Sajiwa Abeywardena in:

Corresponding author

Correspondence to Nuwan Darshana Wickramasinghe.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Work engagement
  • UWES-S
  • Collegiate cycle
  • Sri Lanka
  • Exploratory factor analysis
  • Validity
  • Reliability

BMC Research Notes

ISSN: 1756-0500

Contact us