- Short Report
- Open Access
Demographic differences between health care workers who did or did not respond to a safety and organizational culture survey
BMC Research Notesvolume 4, Article number: 328 (2011)
Areas for institutional improvement to enhance patient safety are commonly identified by surveying health care workers' (HCWs) attitudes, values, beliefs, perceptions and assumptions regarding institutional practices. An ideal response rate of 100% is rarely achieved in such surveys, and non-response bias can occur when non-respondents differ from respondents on a dimension likely to influence survey conclusions. The conditions for non-response bias to occur can be detected by comparing demographic characteristics of respondents and non-respondents and relating any differences to findings in the literature of differences in the construct of interest as a function of these demographic characteristics. The current study takes this approach.
All 5,609 HCWs at a university medical center were invited to participate in a survey measuring safety and organizational culture (response rate = 53.40%). Respondents indicated their professional group, gender, age group, years of working in the hospital and executive function. Because all HCWs were invited, the demographic composition of the group who did not respond was known. Differences in the demographic composition of respondents and non-respondents were compared using separate Pearson's chi-square tests for each demographic characteristic.
Nurses and clinical workers were generally more likely to respond than were physicians, laboratory workers and non-medical workers. Male HCWs were less likely to respond than were females, HCWs aged younger than 45 years old had a lower response rate than did HCWs aged 45 to 54 years old, HCWs who had worked in the hospital for less than 5 years were less likely to respond than were those who had worked in the hospital for 5 years or more and HCWs without an executive function were less likely to respond than were executives.
Demographic characteristics can be linked to response rates and need to be considered in conducting surveys among HCWs. The possibility of non-response bias can be reduced by conducting analyses separately as a function of relevant demographic characteristics, sampling a higher percentage of groups that are known to be less likely to respond, or weighting responses with the reciprocal of the response rate for the respective demographic group.
Patient safety in the hospital depends on health care workers' (HCWs) commitment to safety (i.e., safety culture) [1–9]. Periodic assessment of safety culture is necessary in order to pinpoint attitudes, values, beliefs or perceptions that may need to be changed in order to improve patient safety [1, 3–9]. Safety culture is typically assessed with surveys whose results are then used to design patient safety improvement programs, to evaluate the effectiveness of intervention programs and to track transformations of safety culture over time [1, 3, 4, 8].
A challenge in surveying HCWs is to achieve a high response rate. Physicians, in particular, often show relatively low response rates [1, 10–15]. For example, in a study by Singer et al. , the response rate of physicians was 33%, considerably lower than the response rate of 60% that is recommended for achieving sufficient reliability in the measurement of safety culture . In fact, response rates of 60% or higher are rarely achieved [1, 3]. A major problem with low response rates is that they may result in under- or over-representation of particular groups and, thus, in non-response bias [4, 16–18]. Non-response bias is said to occur when a significant number of those who do not respond differ in terms of relevant characteristics (i.e., characteristics that can influence survey outcomes and conclusions) from those who do respond. Non-response bias is a real concern in health care settings. It has been found as a function of gender with regards to identifying best practices , duration of employment with regards to attitudes towards changes in national health programs , and age with regards to evaluation of alcohol abuse programs .
One way to detect non-response bias is to contact non-respondents personally in order to investigate whether non-respondents' opinions differ substantially from those of respondents [20, 21]. This way of dealing with non-response cannot be applied when surveys are anonymous, as is likely to be the case when HCWs are surveyed [7, 10, 15]. However, when the demographic characteristics of the surveyed population are known, even anonymous surveys can benefit from comparing the demographic composition of respondents with that of non-respondents [12, 16, 19–22].
Although some safety culture surveys conducted in the past have considered the effects of demographic variables on how safety culture is evaluated [2, 3, 6, 7, 23], most studies of safety culture fail to consider the role of demographic characteristics in the evaluation of safety culture or make only vague reference to such differences . An explanation for this might be that the goal of many studies of safety culture is to capture organizational (and not individual) factors that influence patient safety and that the importance of demographic differences in safety culture research has not been recognized . We argue that demographic characteristics should be considered in understanding whether HCWs will be likely to respond to safety culture surveys and that they should be taken into account in reducing the risk of non-response bias for anonymously conducted surveys. The purpose of this study was thus to compare the demographic composition of the groups of HCWs who did or did not respond to a survey measuring safety and organizational culture. Differences in demographic characteristics of respondents and non-respondents that should be taken into account to reduce the risk of non-response bias are documented and suggestions are made for understanding the responding behavior and increasing the response rate of HCWs.
In April 2009, invitations to participate in a safety and organizational culture survey were sent electronically (via intranet) to 5,609 HCWs (out of a total of approximately 8,000 HCWs) involved in patient care (including managers) at the University Medical Center Groningen (UMCG), The Netherlands. The UMCG is a large university medical center that has approximately 1,300 beds, including 53 surgical and medical adult intensive care beds and 46 neonatal and pediatric intensive care beds. Reminders were sent in May and again in June. Participation was on a voluntary basis and no incentives to respond (financial or otherwise) were offered. Anonymity was ensured and informed consent was given by those who responded.
Nine dimensions of safety and organizational culture were assessed in a survey containing 99 items (see Additional File 1). Because the survey addressed general organizational concerns such as work satisfaction, working conditions and perceptions towards the hospital, in addition to aspects of safety culture, it was relevant to all HCWs who were invited to participate. Additional questions addressed department, gender, age, years of working in the hospital and whether one worked in an executive or non-executive function. Because some HCWs could belong to more than one department (e.g., a medical specialist who also teaches at the university belongs to both medical specialist and teaching departments), HCWs were asked to indicate the department where they spent most of their time as stated on their work contract. The ten departments were combined into five professional groups because of similar job descriptions (e.g., the medical specialist and physician assistant departments were combined). The five professional groups thus obtained were "physicians" (e.g., medical specialists and physician assistants), "nurses" (e.g., nurse practitioners and intensive care nurses), "clinical workers" (e.g., dieticians, psychologists and pharmacists), laboratory workers (e.g., laboratory technologists and technicians) and non-medical workers (e.g., facility management workers, secretarial and administrative workers and managers). The age groups used were 15-24 years old, 25-34 years old, 35-44 years old, 45-54 years old and older than 54 years old. The categories used for years of working in the hospital were less than 5 years, 5-9 years, 10-20 years and longer than 20 years. Members of any professional group were considered to hold an executive function if they led a sector, department, unit, subunit or clinic; data were coded according to the number of respondents who worked in an executive or non-executive function.
Because of the need to preserve anonymity, data were available only for the demographic characteristics of the group and not for the conjunctions of the variables for each individual (i.e., we had available to us the data regarding how many, e.g., nurses responded, but not their age distribution, gender, etc., as this would uniquely identify individuals). It was therefore not possible to carry out multivariate analysis to identify which characteristics differed between groups of respondents and non-respondents while taking other characteristics into account. Instead, differences in the demographic characteristics of respondents and non-respondents were tested using Pearson's chi-square tests separately for each demographic characteristic (i.e., professional group, gender, age group, years of working in the hospital, and executive function), with group (respondents vs. non-respondents) and category (e.g., for the characteristic "gender" the categories were male and female) as factors. If interactions of group and category were found (i.e., if the demographic composition of respondents and non-respondents differed), follow-up pairwise Pearson's chi-square tests were conducted between sub-groups. Odds ratios (where an odds ratio of 1 indicates that the demographic composition of respondents and non-respondents did not differ) were used to calculate effect size. A significance level of p < .05, Bonferroni correction for each family of comparisons, was used where necessary.
Out of 5,609 invitations sent, 2,995 were responded to (response rate = 53.40%). Response rates as a function of demographic characteristic are summarized in Table 1.
Chi-square analyses (n = 5609) revealed interactions between group (respondents vs. non-respondents) and each of the demographic characteristics. The Group × Professional Group interaction (X2(4) = 53.54, p < .001) reflects that nurses and clinical workers had significantly higher response rates than did physicians and non-medical workers, and that nurses also had a significantly higher response rate than did laboratory workers (see Table 2 for all follow-up comparisons and odds ratios). The Group × Gender interaction (X2 (1) = 6.33, p < .05) reflects that female HCWs had a higher response rate than did males. The Group × Age Group interaction (X2 (4) = 30.07, p < .001) reflects that HCWs who were younger than 45 years old had significantly lower response rates than did those aged 45 to 54 years old. The Group × Years of Working in the Hospital interaction (X2 (3) = 41.31, p < .001) reflects that HCWs who had worked in the hospital for less than five years had a significantly lower response rate than did those who had worked in the hospital for five years or more. The Group × Executive Function interaction (X2 (1) = 24.77, p < .001) reflects that HCWs with an executive function had a significantly higher response rate than did those without an executive function.
The demographic composition of groups who did or did not respond to a survey of safety and organizational culture was analyzed and significant differences between groups of respondents and non-respondents were found. Response rate was found to depend on professional group, gender, age, years of working in the hospital and executive function.
The survey study on which the current study of demographic differences in response rates is based revealed not only differences in response rates, but differences in how aspects of safety and organizational culture are perceived [Unpublished data of T. A. Listyowardojo, R. E. Nap and A. Johnson]. The existence of differences in response rates as a function of demographic characteristics makes it important to consider whether non-response bias is likely to have influenced the interpretation of the survey results. In our study of safety and organizational culture, the data were analyzed and reported per group, and the major finding of the study was that perceptions of safety and organizational culture differed significantly across professional group, with physicians and non-medical workers tending to give more positive ratings of safety and organizational culture than did nurses, clinical workers and laboratory workers.
The key question addressed in this paper is whether non-response bias due to unequal representation of professional groups influences study results. If group composition is not taken into account when analyzing survey results, under- or over-representation of some groups can influence the conclusions that are drawn. For example, it might be that physicians and non-medical workers, who were relatively positive in their ratings of safety and organizational culture, are less likely to respond to safety culture surveys than are nurses and clinical workers because they feel that changes in institutional practices are not urgently needed. Basing conclusions on the response group as a whole could then lead to an overly negative evaluation of safety culture. More specific aspects of safety culture, such as fear of shame and blame , have been shown to be evaluated differently by nursing professionals than by physicians, and their evaluation may thus also be subject to non-response bias.
The relatively low response rate of physicians may also, in part, be due to the perception that they are too often asked to respond to surveys and that their time is too valuable to be spent completing them [12, 20, 25, 26]. Whereas physicians have been found to complain about being asked to participate in surveys too often, nurses and clinical workers have reported that they are not asked for their professional views often enough . Nurses and the clinical workers may thus embrace the opportunity to voice their points of view by responding to surveys.
The fact that gender affected response rate, with female HCWs being more likely to respond than their male colleagues may be tied to the fact that nurses are predominantly female, and nurses have a higher response rate. Because the effect of one demographic variable could not be statistically isolated from the others, we cannot say that females, in general, were more likely to respond.
The finding that HCWs in the 45 to 54 year old age group were more likely than were younger HCWs to participate contrasts with previous findings of higher response rates for younger HCWs [12, 16, 20, 21]. The fact that these previous studies sampled only physicians (in contrast to the current study, in which all professional groups were surveyed) may be responsible for this difference in findings. Alternatively, it may be that HCWs in this age group were more likely to be senior staff with administrative duties.
The lower response rate of HCWs who had worked in the hospital for less than 5 years as compared to those who had worked for 5 years or more may be related to professional commitment to the hospital, which can be expected to increase as one works longer . A final difference in response rates was that those who had an executive function were twice as likely to participate as were HCWs without an executive function. Those with executive functions may be more willing to participate because they are among those who will use the survey results to develop or defend patient safety intervention programs .
The main limitation of the current study is clearly the survey anonymity. The hospital would release only the demographic characteristics of the group as a whole and those of the group of respondents, and not those of the individuals. Furthermore, because we could not contact non-respondents, we were unable to investigate whether there was a difference in the evaluation of safety and organizational culture between respondents and non-respondents. However, previous studies have reported a strong link between relevant demographic characteristics (i.e., professional groups, age, years of work experience and executive function) and safety attitudes [1–3, 6, 24, 27, 28], making it possible to draw tentative conclusions about the possibility of non-response bias.
Demographic characteristics can be linked to response rates and thus need to be taken into account in conducting surveys among HCWs. The possibility of non-response bias can be reduced by conducting analyses separately as a function of relevant demographic characteristics or by sampling a higher percentage of members of groups that are known to be less likely to respond [1–3]. Another approach to reducing the possibility of non-response bias is to weight responses with the reciprocal of the response rate for the respective demographic group [1, 29].
Singer SJ, Gaba DM, Geppert JJ, Sinaiko AD, Howard SK, Park KC: The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care. 2003, 12: 112-118. 10.1136/qhc.12.2.112.
Singer SJ, Falwell A, Gaba DM, Meterko M, Rosen A, Hartmann CW, Baker L: Identifying organizational cultures that promote patient safety. Health Care Manage Rev. 2009, 34: 300-311.
Singer SJ, Gaba DM, Falwell A, Lin S, Hayes J, Baker L: Patient safety climate in 92 US hospitals differences by work area and discipline. Med Care. 2009, 47: 23-31. 10.1097/MLR.0b013e31817e189d.
Nieva VF, Sorra J: Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care. 2003, 12: 17-23.
Pronovost P, Sexton B: Assessing safety culture: guidelines and recommendations. Qual Saf Health Care. 2005, 14: 231-233. 10.1136/qshc.2005.015180.
Pronovost PJ, Weast B, Holzmueller CG, Rosenstein BJ, Kidwell RP, Haller KB, Feroli ER, Sexton JB, Rubin HR: Evaluation of the culture of safety: survey of clinicians and managers in an academic medical center. Qual Saf Health Care. 2003, 12: 405-410. 10.1136/qhc.12.6.405.
Huang DT, Clermont G, Sexton JB, Karlo CA, Miller RG, Weissfeld LA, Rowan KM, Angus DC: Perceptions of safety culture vary across the intensive care units of a single institution. Am J Respir Crit Care Med. 2007, 35: 165-176.
Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ: The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006, 6:
Antonsen S: Safety Culture: Theory, Method, and Improvement. 2009, Surrey: Ashgate Publishing Limited
Asch DA, Jedrziewski MK, Christakis NA: Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997, 50: 1129-1136. 10.1016/S0895-4356(97)00126-1.
Asch S, Connor SE, Hamilton EG, Fox SA: Problems in recruiting community-based physicians for health services research. J Gen Intern Med. 2000, 15: 591-599. 10.1046/j.1525-1497.2000.02329.x.
Cartwright A: Professionals as responders - variations in and effects of response rates to questionnaires, 1961-77. Br Med J. 1978, 2: 1419-1421. 10.1136/bmj.2.6149.1419.
Cummings SM, Savitz LA, Konrad TR: Reported response rates to mailed physician questionnaires. Health Serv Res. 2001, 35: 1347-1355.
Ward J: General-practitioners experience of research. Fam Pract. 1994, 11: 418-423. 10.1093/fampra/11.4.418.
VanGeest JB, Johnson TP, Welch VL: Methodologies for improving response rates in surveys of physicians - a systematic review. Eval Health Prof. 2007, 30: 303-321. 10.1177/0163278707307899.
Cull WL, O'Connor KG, Sharp S, Tang SFS: Response rates and response bias for 50 surveys of pediatricians. Health Serv Res. 2005, 40: 213-226. 10.1111/j.1475-6773.2005.00350.x.
de Vaus DA: Finding a sample. Surveys in social research. 1996, NSW: Allen and Unwin Pty Ltd., 60-79. Fourth
Guadagnoli E, Cunningham S: The effects of nonresponse and late response on a survey of physician attitudes. Eval Health Prof. 1989, 12: 318-328. 10.1177/016327878901200306.
McFarlane E, Olmsted MG, Murphy J, Hill CA: Nonresponse bias in a mail survey of physicians. Eval Health Prof. 2007, 30: 170-185. 10.1177/0163278707300632.
Armstrong D, Ashworth M: When questionnaire response rates do matter: a survey of general practitioners and their views of NHS changes. Br J Gen Pract. 2000, 50: 479-480.
Templeton L, Deehan A, Taylor C, Drummond C, Strang J: Surveying general practitioners: does a low response rate matter?. Br J Gen Pract. 1997, 47: 91-94.
Bjertnaes OA, Garratt A, Botten G: Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians. Eval Health Prof. 2008, 31: 65-80.
Johnson A, Bakker S: Cooperation and safety culture in intensive care units and safety and rescue teams. Tijdschrift voor Ergonomie. 2007, 32 (2): 22-23.
Cooper MD, Phillips RA: Exploratory analysis of the safety climate and safety behavior relationship. J Safety Res. 2004, 35: 497-512. 10.1016/j.jsr.2004.08.004.
Sudman S: Mail surveys of reluctant professionals. Eval Rev. 1985, 9: 349-360. 10.1177/0193841X8500900306.
Kaner EF, Haighton CA, Mcavoy BR: 'So much post, so busy with practice - so, no time!': a telephone survey of general practitioners' reasons for not participating in postal questionnaire surveys. Br J Gen Pract. 1998, 48: 1067-1069.
Lin SH, Tang WJ, Miao JY, Wang ZM, Wang PX: Safety climate measurement at workplace in China: a validity and reliability assessment. Saf Sci. 2008, 46: 1037-1046. 10.1016/j.ssci.2007.05.001.
Sexton JB, Holzmueller CG, Pronovost PJ, Thomas EJ, McFerran S, Nunes J, Thompson DA, Knight AP, Penning DH, Fox HE: Variation in caregiver perceptions of teamwork climate in labor and delivery units. J Perinatol. 2006, 26: 463-470. 10.1038/sj.jp.7211556.
Holt D, Elliot D: Methods of weighting for unit nonresponse. Statistician. 1991, 40: 333-342. 10.2307/2348286.
The authors declare that they have no competing interests.
TAL contributed to study conception and design, data analysis and interpretation, drafting and critically revising the manuscript. REN contributed to study conception and design and acquisition of data. AJ contributed to data analysis and interpretation, and drafting and critically revising the manuscript. All authors read and approved the final manuscript.