Veterinary students’ perceptions of their learning environment as measured by the Dundee Ready Education Environment Measure

Abstract

Background

The Dundee Ready Education Environment Measure (DREEM) has been widely used to evaluate the learning environment within health sciences education, however, this tool has not been applied in veterinary medical education. The aim of this study was to evaluate the reliability and validity of the DREEM tool in a veterinary medical program and to determine veterinary students’ perceptions of their learning environment.

Methods

The DREEM is a survey tool which quantitatively measures students’ perceptions of their learning environment. The survey consists of 50 items, each scored 0–4 on a Likert Scale. The 50 items are subsequently analysed within five subscales related to students’ perceptions of learning, faculty (teachers), academic atmosphere, and self-perceptions (academic and social). An overall score is obtained by summing the mean score for each subscale, with an overall possible score of 200.

All students in the program were asked to complete the DREEM. Means and standard deviations were calculated for the 50 items, the five subscale scores and the overall score. Cronbach’s alpha was determined for the five subscales and overall score to evaluate reliability. Confirmatory factor analysis was used to evaluate construct validity.

Results

224 responses (53%) were received. The Cronbach’s alpha for the overall score was 0.93 and for the five subscales were; perceptions of learning 0.85, perceptions of faculty 0.79, perceptions of atmosphere 0.81, academic self-perceptions 0.68, and social self-perceptions 0.72. Construct validity was determined to be acceptable (p < 0.001) and all items contributed to the overall validity of the DREEM. The overall DREEM score was 128.9/200, which is a positive result based on the developers’ descriptors and comparable to other health science education programs. Four individual items of concern were identified by students.

Conclusions

In this setting the DREEM was a reliable and valid tool to measure veterinary students’ perceptions of their learning environment. The four items identified as concerning originated from four of the five subscales, but all related to workload. Negative perceptions regarding workload is a common concern of students in health education programs. If not addressed, this perception may have an unfavourable impact on veterinary students’ learning environment.

Background

Veterinary medical education in the United States is currently facing a paradigm shift with calls for greater emphasis on competency-based education. These competencies include a range of professional or non-technical attributes to address heightened expectations of the veterinary profession [1–3]. This broadening of veterinary curricula is in the face of rapidly expanding knowledge across all animal species for which veterinarians are responsible. As a result, many veterinary curricula have become increasingly demanding, with associated increases in workload and intensity often culminating in elevated stress and lack of motivation in veterinary students [4, 5]. However, more in-depth appraisal of students’ perceptions of the current learning environment have not been performed in veterinary programs to identify specific areas of concern. In health sciences education, the environment in which students learn has been shown to influence a number of educational outcomes including academic achievement, course satisfaction, student aspirations, as well as sense of well-being [6–11]. In these settings the Dundee Ready Education Environment Measure (DREEM) is the most common tool used to evaluate students’ perceptions of their learning environment [12]. For example, this tool has been used to assess the impact of curricular change and interventions [13–15], as well as identifying strengths and weakness of curricula with a view to introducing change [16, 17].

The DREEM is a 50-item questionnaire which measures students’ overall perceptions of their learning environment as well as five different focus areas or subscales; students’ perceptions of learning, students’ perceptions of course organizers (faculty), students’ academic self-perceptions, students’ perceptions of atmosphere, and students’ social self-perceptions. This tool has been translated into at least ten languages [18] and applied in multiple healthcare professional settings in over twenty countries [19]. The different healthcare professional programs where this instrument has been utilized includes medical (undergraduate and postgraduate), dental, nursing, and chiropractic [20–24], but no veterinary programs have been investigated to date. This is despite the fact that veterinary medical programs have many similarities in their structure and degree of difficulty to other healthcare programs, especially human medicine. However, when using an evaluation instrument in a different context it is important to determine its reliability and validity in the new environment. Furthermore, inconsistent reporting of statistical analyses, and variable results in other studies, has led to calls for further psychometric evaluation of the DREEM instrument [12, 19].

We were also interested in applying this instrument in our educational setting as a number of factors inherent to United States veterinary educational systems could influence students’ perceptions of their learning environment and influence DREEM results. For example, veterinary education is predominantly post-graduate with the majority of students having completed at least 60 credit hours of undergraduate coursework. This is in contrast to undergraduate medical programs outside the United States in which students may enter immediately following high school. In addition, the gender ratio in veterinary programs is approximately 80% female, while in medical schools this ratio is usually lower. Students’ perceptions also may be influenced by age, where the average age of students entering our program is 24, considerably older than students in undergraduate medical programs. Furthermore, many of our students are pursuing veterinary medicine as a second career, again in contrast to undergraduate medical education programs. For these reasons we wished to evaluate the DREEM in a veterinary medical program in the United States.

The aims of this study were; 1) to evaluate the psychometric properties of the DREEM in the context of veterinary medical education, and 2) to determine students’ perceptions of their learning environment within a United States veterinary program and identify potential areas of student concern.

Methods

Doctor of Veterinary Medicine (DVM) program

The Virginia Maryland Regional College of Veterinary Medicine (VMRCVM) is an accredited veterinary program within the United States. The College offers a four year, Doctor of Veterinary Medicine (DVM) degree with most students starting after completion of a four year undergraduate degree. The program comprises three years (six semesters) of pre-clinical training and a final year of work place-based training in a variety of clinical settings including a veterinary teaching hospital. The VMRCVM has a traditional curriculum (non-problem based learning) with additional courses being offered to allow students to focus in areas of interest (tracks) above and beyond the core curriculum. However, as there is no limited licensure within veterinary practice, students must demonstrate competencies within each major animal species area. There are a total of 419 students within the DVM program, approximately 80% being female, with an average age of 24 upon entry.

Participants and data collection

In January 2013, all students (n = 419) enrolled in the DVM program were invited to participate in a self-administered, on-line DREEM survey. The survey remained available for one week. An email to the students was used to convey background information on the DREEM, instructions for completing the survey, and how results would be used. Anonymity of the survey was emphasized. Prior to their participation, students were invited to attend an optional twenty minute presentation to provide further explanation of the DREEM tool and its purpose. Attendance was not monitored at this presentation. The study was approved by the Office of Research Compliance at Virginia Tech (IRB#12-957).

Dundee ready education environment measure

The DREEM is a survey tool developed to quantitatively measure student’s perceptions of their learning environment within a health profession educational setting [18]. The DREEM survey consists of 50 items, or statements, each scored 0–4 on a 5-point Likert Scale (0 = strongly disagree to 4 = strongly agree) [25]. There are nine negatively stated items (questions 4, 8, 9, 17, 25, 35, 39, 48 and 50) which are scored in reverse (0 = strongly agree to 4 = strongly disagree). Mean scores for individual items are calculated with a maximum score of 4.0 for each item. The 50 items are subsequently analysed within five subscales created by combining specific items [25]; 1) students’ perceptions of learning (12 items, maximum score 48), 2) students’ perceptions of faculty (11 items, maximum score 44), 3) students’ academic self-perceptions (8 items, maximum score 32), 4) students’ perceptions of atmosphere (12 items, maximum score 48) and 5) students’ social self-perceptions (7 items, maximum score 28). An overall or composite score is obtained by summing the mean score for each subscale, with a maximal possible score of 200.

An approximate guide to interpretation of the mean overall score, the subscale scores, and scores for individual items has been established by the developers of the DREEM (Table 1) [26].

Table 1 Interpretation of mean scores as defined by the DREEM developers[26]

In this study, the DREEM survey was used in its original language with the exception of two terms which were not familiar to veterinary students in the United States. “Course organisers” was substituted with “faculty”, and “registrar” was replaced by “student”.

Statistical analysis

Normal probability plots showed that scores for each of the items, composite scores of the five subscales, and the overall composite score followed an approximately normal distribution. For the individual items, scores increased in a step-plot pattern without showing minimum or maximum inflation. For the composite scores, near straight lines were observed. Subsequently, means and standard deviations were calculated for all 50 items as well as the five subscales and the overall score for the entire student sample.

The parametric measure of internal consistency (Cronbach’s alpha) was determined for the overall score and each subscale using PROC CORR. The following guide for interpretation of Cronbach’s alpha was used; α ≥ .90 = excellent, 0.70-0.89 = good, and 0.60-0.69 = acceptable [27]. An acceptable or higher Cronbach’s alpha demonstrates that the individual items that constitute a test, the DREEM tool in this case, correlate well with one another or with the test total.

To examine construct validity of the DREEM in this setting a confirmatory factor analytic model was fitted to the data using PROC CALIS. This analysis was used to measure whether the construct was consistent with a hypothesized measurement model. In our model we specified the five subscales as latent variables (structural variables) and the 50 items as manifest variables (indicator variables). Model parameters were estimated using maximum likelihood. Four criteria [28] were inspected to assess model fit; the ratio of goodness of fit chi-square to the degrees of freedom, the Bentler Comparative Fit Index, the Bentler-Bonett Non-normed Index, and inspection of the standard errors for the loading factors to verify that none had a value near 0.

Finally, standardized and unstandardized t values were examined to assess whether the indicator variables were measuring the underlying constructs of interest (i.e., perceptions of learning, perceptions of faculty, academic self-perceptions, perceptions of atmosphere, and social self-perceptions). All analyses were performed using SAS version 9.3 (Cary, NC, USA).

Results

There were 224 respondents who fully completed the survey (53% response rate), with 179 females and 45 males which reflect the gender distribution of our student population.

Statistical analysis of the collected data determined the internal consistency (Cronbach’s α) for the overall score to be 0.93, and for the five subscales; 1) students’ perceptions of learning = 0.85, 2) students’ perceptions of faculty = 0.79, 3) students’ academic self-perceptions = 0.68, 4) students’ perceptions of atmosphere = 0.81, and 5) students’ social self-perceptions = 0.72 (Table 2). Results for confirmatory factor analysis of the construct validity are recorded in Table 3. Based on the model fit criteria evaluated, we concluded the model fit was adequate. Unstandardized and standardized t values for all 50 items were greater than 3.291 and therefore significant at p < 0.001 [28]. All items contributed to the overall validity of the DREEM.

Table 2 Results for the overall DREEM score and the five subscale scores

Table 3 Confirmatory factor analysis of the DREEM by subscale

The overall score for the DREEM was 128.9 out of a possible 200 and the five subscale scores were; 1) students’ perceptions of learning 30.5 out of 48, 2) students’ perceptions of faculty 31.3 out of 44, 3) students’ academic self-perceptions 20.2 out of 32, 4) students’ perceptions of atmosphere 29.9 out of 48, and 5) students’ social self-perceptions 17.1 out of 28. The overall score and subscale scores, together with the interpretative descriptors recommended by the developers [26] are recorded in Table 2. Comparison of the overall score in the current study with those observed in other health science educational programs are summarized in Table 4.

Table 4 Comparison of the overall DREEM scores of health science educational programs

The means and standard deviations for individual items of the DREEM are recorded in Table 5. Items were considered areas of concern if the mean item score was ≤ 2.0 [26]. Based on this threshold, four items were identified; “I am too tired to enjoy the program” (mean 1.79 ± 1.03), “the program is well scheduled” (mean 1.39 ± 0.99), “the teaching overemphasizes factual learning” (mean 1.67 ± 0.96), and “I am able to memorize all I need” (mean 1.31 ± 0.98).

Table 5 Mean and standard deviation for individual items of the DREEM

Discussion

Over the last four decades there has been growing interest in students’ perceptions of their educational environment and the impact this may have on subsequent learning [19, 25, 29]. Educational research has demonstrated that the learning environment may influence student behaviour, academic achievement, course satisfaction and aspirations, as well as sense of well-being [6–11]. Therefore, evaluation of the educational environment is an important consideration for any programmatic or curricular review [12, 14].

To our knowledge, this is the first time the DREEM has been applied within a veterinary educational setting and the first time within a United States healthcare professional program. Previously the DREEM has predominantly been used to evaluate the learning environments in undergraduate medical education programs in a variety of countries [19], but not the United States. Although medical and veterinary educational programs have many similarities, there are sufficient unique aspects of veterinary education to warrant investigation of the learning environment in this context. To this end, we wished to evaluate the internal reliability and construct validity of the overall DREEM tool and the five subscales, as well as identify items of concern as perceived by veterinary students.

In this study, the DREEM was shown to be a reliable tool in which to measure perceptions of the learning environment in a sample of veterinary students. Internal consistency was “excellent” [27] for the overall score of the DREEM with a Cronbach’s alpha = 0.93. This result is similar to the Cronbach’s alpha of the overall score in other studies which range from 0.87 - 0.93 [12, 20, 30–34]. The internal consistency of the subscales in this study were either “excellent” or “good” based on accepted guidelines [27], with the exception of students’ perception of academic-self which had an “acceptable” result of 0.67. The reasons for the lower internal consistency for academic self-perception can only be speculated, but may be due to the variable daily academic experiences of veterinary students affecting the consistency of their answers. Interestingly, other studies report more variable internal consistencies for the five subscales ranging from “good” (e.g., 0.80) to “poor” (e.g., 0.47), but no consistent pattern is observed [12, 20, 30–34]. This inconsistent reliability of the subscales in other studies has resulted in calls for analysis of the validity of the instrument [12].

The construct validity of the DREEM within this study was determined to be acceptable by confirmatory factor analysis, although the goodness of fit for the model was weak. Repetition of the study in our setting, as well as other veterinary programs, may allow better evaluation of the construct validity for the DREEM tool in veterinary medical education. Furthermore, repetition would allow better assessment of individual items in the tool, including whether they should be omitted or re-written, potentially strengthening construct validity. Other studies have evaluated the psychometric properties of the DREEM and found variable results for construct validity, as well as subscale reliability [12, 20, 30–34]. This variation between studies emphasizes the need for continued psychometric evaluation of the DREEM to better determine its suitability for evaluation of students’ perceptions of their educational environment, as well as comparison of these environments in a variety of health education settings.

After the internal consistency and construct validity of the DREEM was determined in this study, descriptive interpretations of quantitative results were applied as outlined by the developers of the tool [26]. Using these recommendations, the overall score of 128.9 observed in the current study can be interpreted as “more positive than negative”. While this is below the highest designation for this category (Table 1), our overall score for students’ perceptions was above the median overall score for studies conducted in medical, dental, nursing and chiropractic programs (Table 4). As with the overall score, the interpretation of subscale scores for veterinary student perceptions’ were positive in all five subscales based on the developers’ descriptors (Table 2). Comparison of these results to other studies is more difficult as there are no consistent findings for specific subscales between studies, with positive and negative results recorded for all subscales across a variety of health sciences programs.

A threshold mean score of ≤2.0 is used to interpret individual items based on recommendations of the DREEM developers. Using this threshold, four items of concern were identified in this study. The four items were; (1) “I am unable to memorize all I need” (mean = 1.31 ± 0.98), (2) “the program is well scheduled” (mean = 1.39 ± 0.99), (3) “the teaching over emphasises factual learning” (mean = 1.67 ± 0.96), and (4) “I am too tired to enjoy the program” (mean = 1.79 ± 1.03). It is important to note that these four items are derived from different subscales, but all relate to a common theme of workload. Furthermore, this finding supports earlier studies describing concerns about expanding curricula in veterinary medical education and the impact on student learning [5, 35, 36]. These results are worrying, as factors impeding the academic progress of veterinary students may include lack of motivation, stress, and fatigue [36]. These factors all can be related to a curriculum’s heavy workload and intensity [35]. A further consequence of this perceived heavy workload may be “reproduction of information rather than construction of knowledge” [5]. Interestingly, students in medical and dental programs have similar concerns regarding their ability to “memorize all they need” as reported in other DREEM studies [20, 37]. It therefore appears that a consistent theme within healthcare education is that workload is perceived as being excessive and this perception may have a negative impact on the learning environment.

Solutions to decrease curricula load in veterinary medical education could involve deleting redundant content by focusing on Day-1 competencies; consideration of limited licensure (i.e., veterinarians licensed to practice on fewer species such as dogs/cats only) allowing for decreased curricula content; or lengthening the time of training to include an internship year during which students are not licensed to practice. Each solution has advantages and disadvantages but all would lead to widespread curricula reform and would require agreement amongst major stakeholders including the American Veterinary Medical Association as well as veterinary testing, licensing, and accreditation bodies within the United States.

A further consideration regarding students’ negative perceptions of scheduling is that these concerns are likely multifactorial and could relate to contact hours, timing of assessments, errors in timetables, or equity of schedules for different groups. In our situation, schedules have been created manually which has allowed for multiple errors in the schedule. Based on students’ feedback, this appeared to be a major source of concern and frustration. Although this problem has been addressed with the introduction of an electronic scheduling program, it highlights the negative impact administrative processes may have on the learning environment.

One potential issue with the current study is the relatively low response rate (53%) achieved. Variable response rates have been reported in other DREEM studies ranging from 48% to 97% [38, 39]. However, many studies do not report a response rate, rather the number of students responding to the survey, making comparison of response rates difficult. The response rate in the current study was impacted by the low participation of students in the final year where these students are undergoing their clinical training and are therefore distributed around the United States. Furthermore, the lower response rate of the students in the clinical year may have negatively impacted the DREEM scores, as these students frequently have a more positive outlook on their educational environment due to their immersion in workplace-based learning. To address this issue we intend to repeat this survey in the coming year and will administer the survey immediately before graduation when all final year students have returned to the college. In addition, the DREEM survey will be administered to students in the pre-clinical years as part of a course which should improve their response rate.

Conclusions

This study has demonstrated that the DREEM can be applied in a veterinary medical context to provide reliable and valid information on veterinary students’ perceptions of their learning environment. Furthermore, the study identified student concerns regarding workload. As veterinary educators, it is interesting to note that other health education programs encounter similar challenges and concerns in their learning environments. This highlights the utility in sharing information and solutions regardless of the specific program. Similarly, we believe it would be beneficial for other veterinary programs worldwide to apply the DREEM to allow comparison between programs in this field. Sharing this information could lead to better development of best practices in veterinary student learning environments.

Authors’ information

JMP is the Director of Admissions and Student Services at the Virginia Maryland Regional College of Veterinary Medicine (VMRCVM). Her interests include teaching and assessing professional competencies and admissions policies and procedures. JLH is the Associate Dean for Professional Programs at the VMRCVM. Her interests include curriculum design and program development. SRW is a Research Assistant Professor at the VMRCVM. His interests include psychometric analysis of research data.