Effect of the e-flipped learning approach on the knowledge, attitudes, and perceived behaviour of medical educators
BMC Research Notes volume 15, Article number: 227 (2022)
Using the Kirkpatrick evaluation model, investigate the effect of the e-flipped learning approach on the knowledge, attitudes, and perceived behaviour of medical educators.
This interventional study was conducted on 140 eligible medical educators at Shiraz University of Medical Sciences (SUMS) from 2019 to 2021. A researcher-made questionnaire was developed to assess the impact of the course on three levels of reaction, knowledge, and perceived behaviour using the Kirkpatrick evaluation model. According to the findings, the average reaction and knowledge scores are higher than the desired level, but the perceived behaviour score is slightly lower than the desired level. The e-flipped learning approach can improve medical educators’ knowledge, attitude, and behaviour regarding online teaching.
The widespread prevalence of COVID-19 pandemic disease has challenged educational environments. In this regard, universities use new technologies to provide their students with much more accessibility . Furthermore, e-learning has become the mainstream of education in times of crisis. As the demand for online education increases, the need to improve the quality of online teaching is felt . Providing quality educational programs to medical educators will be possible when educators are familiar with new technologies and apply appropriate online learning strategies . Evidence shows that active and interactive learning methods, like flipped learning, help educators improve their teaching skills and use the theoretical knowledge they've learned in the real world .
During the coronavirus outbreak, flipped learning combined with online teaching generated a new learning paradigm known as the “e-flipped learning model” that promotes the online approach’s efficacy . This strategy combines asynchronous and synchronous learning, including pre-class, in-class, and post-class learning activities. Before attending live classes, students are exposed to the material through asynchronous multimedia content. Synchronous in-class activities are typically facilitated discussions to enhance the students’ cognitive abilities. Finally, the performance of students is evaluated following the live classes. In the e-flipped paradigm, these phases should be carefully put together to get the intended learning results .
In addition, the quality of educational intervention should be evaluated according to appropriate standards to enable their continuous improvement . Evaluating these courses and their outcomes requires an applied evaluation technique to assess the level of participants’ knowledge, attitude, and skills acquired after the course .
The Kirkpatrick evaluation model assesses educational programs: reaction, learning, behaviour, and outcome . Reaction (learner response to the training experience), learning (following education, how much the participants learned about the subject matter). This level is typically assessed using a pretest and posttest, behaviour (whether the learning was applied in the workplace), and four outcomes (the long-term impact of training). The first three levels of the Kirkpatrick evaluation model have been assessed in some form in most academic programs. The fourth level looks at the program's long-term effects on society and requires long-term follow-up in the community [9,10,11,12].
Most medical educators provided online education throughout the COVID-19 period so that an e-flipped learning approach could improve the quality of their teaching . In addition, no research has been done on educating medical educators through the e-flipped learning approach at SUMS, and an information gap is felt in this field. As a result, the present study was conducted to investigate the effect of the e-flipped learning approach on the knowledge, attitudes, and perceived behaviour of medical educators in online education.
The present study is a one-group interventional study.
The inclusion criteria were all educators affiliated with SUMS who enrolled in the educational intervention based on the e-flipped learning approach, with a minimum education level of a doctorate, willingness to participate, and an informed consent form. All those unwilling to continue their cooperation received similar courses, and those with extended leave of absence during the study were excluded.
This intervention included developing educational content, selecting a strategy, and implementing the intervention as follows:
This e- flipped educational intervention targets medical educators involved in online teaching.
To increase medical educators’ knowledge, attitude, and perceived behaviour about online teaching.
This intervention is based on constructivism, which holds that the learner constructs knowledge by doing and interacting with others , self-directed learning, which emphasizes students’ responsibility for their education , and multimedia learning to promote active and interactive learning through a variety of media such as text, audio, images, animations or videos . Therefore, designing learning environments based on these theories results in an interactive, active, and flexible endeavor considered in our research.
Twenty multimedia content (e.g., brief videos, podcasts, power points) focused on the following topics: concepts and principles of e-learning, how to become an e-teacher, designing technology-enabled learning, online assessment methods and tools, and innovative technology in medical education.
This course was conducted in the following manner by the e-flipped learning method :
First, the participants were assigned the Learning Management System (LMS), read the course content, and completed the pretest, and then the educational materials were opened for them. Before joining the live sessions, participants read asynchronous content or performed learning activities such as dialogue with peers on a group discussion board. The Learners were guided step-by-step using icons embedded in the learning path. Eight three-hour online sessions were held over 8 weeks. First, the instructors provided learning goals, short presentations, and open-ended questions. Participants had small group discussions in eight virtual breakout rooms. The instructors engaged as facilitators, guiding and encouraging individuals to participate. The lecturers reinforced the examples using PowerPoint and educational videos and then summarized the topics.
A forum for asynchronous discussion and a virtual classroom for synchronous interaction was used. Also, a social network was designed as a second method to answer the participants' questions. It involved using a second platform, such as social media, to allow interaction and engagement. The participants were also encouraged to share their experiences and additional resources on the WhatsApp group.
Five e-learning professors and two physicians with 15 years of work experience in this field were responsible for teaching the course. Also, four information technology experts and two content development experts were active as technical and educational support.
These courses were held through LMS over 8 weeks, including 20 asynchronous and eight online synchronous sessions.
Data collection tools
The faculty members' knowledge, attitudes, satisfaction, and perceived behaviours about online education were evaluated at baseline and 1 week after finishing the training sessions. A researcher-made questionnaire was used based on the first to third levels of the Kirkpatrick evaluation model (additional file 1). The questionnaire consists of five parts:
1-Demographic information (age, sex, education, employment status, job duration).
Level 1. Reaction: assessing the participants’ attitude and satisfaction towards online teaching and the training course at the end of the program (41 questions).
Level 2: Knowledge: evaluating knowledge level before and after the course (12 questions).
Level 3: Perceived behaviour: evaluating the participants' performance after the course (14 questions).
The questions were designed based on the five-point Likert scale. In each dimension of the questionnaire, Scores higher than 60% were considered desirable.
Validity and reliability
Ten experts on medical education, e-learning, and educational management were invited to comment on the questionnaire's aesthetics and content validity. The questionnaire's content validity index (CVI) was calculated on the attitude (0.86), the satisfaction (0.95), the knowledge (0.96), and the behavior (0.96) dimensions. Cronbach's alpha confirmed the reliability of the questionnaire on the attitude (0.90), satisfaction (0.94), knowledge (0.95), and perceived behaviour (0.93) dimensions.
SPSS 21 was used to analyze the collected data, including descriptive and analytical statistical tests. A paired t-test was used for comparing pre and posttest knowledge scores. Also, ANOVA and an independent-sample t-test were used to compare the differences across the socio-demographic groups. The Pearson correlation coefficient was used to examine the relationship between the three levels of the questionnaires. A P-value of 0.05 was considered the significance level.
One hundred and eighty faculty members of the Shiraz University of Medical Sciences participating Sciences who enrolled in the e-flipped course were selected by census method from July 2019 to December 2021. Each course lasted 8 weeks and had 20–22 participants.
Of the 180 eligible participants who started the study, 140 (79%) professors completed it. Of those, 67.9% were women, 32.1% were men, 70% were assistant professors, and 66.4% had less than 10 years of work experience. Table 1 shows a comparison of pre-and post-intervention knowledge subscale scores.
The paired t-test showed that all subscales’ knowledge scores after the intervention were significantly higher. Regarding the results of the Pearson’s correlation coefficient analysis, a significant relationship was reported between the knowledge scores before and after training (p < 0.0001) (Fig. 1).
As shown in Table 2, the attitude and satisfaction mean scores are higher than the desired level (> 60%). However, the perceived behaviour score is slightly lower than the desired level. Also, the result showed that there was a significant relationship between participants' perceived behaviour, attitude, and satisfaction.
The analysis of variance test (ANOVA) and an independent-sample t-test showed no significant relationship between participants' demographic variables and knowledge, attitudes, satisfaction, and perceived behaviour (Additional file 2).
This study aimed to assess the efficacy of e-flipped approach on the knowledge, attitudes, and perceived behaviour among medical educators in online education using Kirkpatrick’s model. According to the findings, this approach increased participants’ knowledge, attitude, and perceived behaviour towards online education. The results of the present study are consistent with the results of studies of similar teaching interventions on the level of learning and retention [12, 13], knowledge, attitude and satisfaction , and knowledge and skill  in university educators.
On the other hand, some studies show the same effectiveness for online and face-to-face educational courses . Effectiveness, according to some experts, is closely connected to instructional strategies. Thus, in general, the online or face-to-face presentation method alone does not make a difference in the audience's learning . In the present study, we used active and interactive learning methods, one of which was small group discussion. Alrehaili et al. believe that interactive learning and role-playing can capture the challenges of natural environments and enhance deep learning, intrinsic motivation, and mastery of learning .
Also, by providing asynchronous multimedia, the participants could study flexibly concerning time, location, and pace. Multimedia that uses the right combination of media, such as text, shape, image, video, and audio, is very appealing to learners with different learning styles, draws their attention to educational content, and improves interaction with content . Furthermore, interaction with other participants was strengthened by creating group discussions synchronously through virtual breakout rooms, a forum, and a WhatsApp group. These methods support learners' interaction with classmates and teachers. Learners have different preferences for learning that can be met by using various teaching methods and improving their attitude towards educational content . It also allows learners to learn with the flexibility of time and place and learning speed . This way, people can apply a piece of knowledge or skill with practice and repetition. In this regard, the e-flipped approach can be used to educate medical educators.
The study also faced some limitations, including the following: (1) the behavioural items were self-reported responses, which could be subject to significant bias; (2) the sample came from only one university; (3) the faculty in the sample were already involved in online teaching; (4) the evaluation was a pre-post design with an evaluation that occurred 1 week after the training was completed. Given these limitations, future research should evaluate the e-flipped learning approach over a more extended period and with larger sample size.
Availability of data and materials
The data supporting this study's findings are available from the corresponding author on request.
Content validity index
Shiraz University of Medical Sciences
Learning Management System
Daniel SJ. Education and the COVID-19 pandemic. Prospects. 2020. https://doi.org/10.1007/S11125-020-09464-3:91-6.
Fataar A, Norodien-Fataar N. Towards an e-learning ecologies approach to pedagogy in a post-COVID world. J Educ. 2021;2021(84):155–68.
Calderón A, Scanlon D, MacPhail A, Moody B. An integrated blended learning approach for physical education teacher education programmes: teacher educators’ and pre-service teachers’ experiences. Phys Educ Sport Pedagog. 2020;26(6):562–77. https://doi.org/10.1080/17408989.2020.1823961.
Ye XD, Chang YH, Lai CL. An interactive problem-posing guiding approach to bridging and facilitating pre- and in-class learning for flipped classrooms. Interact Learn Environ. 2018;27(8):1075–92. https://doi.org/10.1080/10494820.2018.1495651.
Guraya S. Combating the COVID-19 outbreak with a technology-driven e-flipped classroom model of educational transformation. J Taibah Univ Med Sci. 2020;15(4):253.
Kularathne SA. Teacher empowerment on second language education, a malnourished baby. SSRN Electron J. 2021. https://doi.org/10.2139/ssrn.3808913.
Murillo-Zamorano LR, López Sánchez JÁ, Godoy-Caballero AL. How the flipped classroom affects knowledge, skills, and engagement in higher education: effects on students’ satisfaction. Comput Educ. 2019;1(141): 103608.
Kusumaningrum H, Syahrial Z, Erwin TN. Measuring e-learning effectiveness of the standard operating procedure course using the 1st and 2nd levels of Kirkpatrick model. Int J Nov Res Educ Learn. 2022;5:82–8.
Van Hulle VC. Nurses’ perceptions of children’s pain: a pilot study of cognitive representations. J Pain Symptom Manage. 2007;33(3):290–301.
Campbell K, Taylor V, Douglas S. Effectiveness of online cancer education for nurses and allied health professionals; a systematic review using kirkpatrick evaluation framework. J Cancer Educ. 2019;34(2):339–56. https://doi.org/10.1007/s13187-017-1308-2.
Jones C, Fraser J, Randall S. The evaluation of a home-based paediatric nursing service: concept and design development using the Kirkpatrick model. J Res Nurs. 2018;23(6):492–501. https://doi.org/10.1177/1744987118786019.
Jeppesen A, Joyce B. Challenges and successes faculty reflections on a college teaching development program. Collect Essays Learn Teach. 2018. https://doi.org/10.22329/celt.v11i0.4972.
Sairenji T, Jarrett JB, Baldwin LM, Wilson SA. Overview and assessment of a full-time family medicine faculty development fellowship. Fam Med. 2018;50(4):275–82.
Shafi R, Rauf A, Irshad K, Zaman I, Hussain A, Shah SS, et al. Faculty development program: way to excellence. J Shifa Tameer-e-Millat Univ. 2020;3(2):107–12.
Alzahrani FY, Althaqafi AS. EFL teachers’ perceptions of the effectiveness of online professional development in higher education in Saudi Arabia. High Educ Stud. 2020;10(1):121–31.
Traeger AC, Lee H, Hübscher M, Skinner IW, Moseley GL, Nicholas MK, et al. Effect of intensive patient education vs placebo patient education on outcomes in patients with acute low back pain: a randomized clinical trial. JAMA Neurol. 2019;76(2):161–9.
Alrehaili EA, Al OH. A virtual reality role-playing serious game for experiential learning. Interact Learn Environ. 2019. https://doi.org/10.1080/10494820.2019.1703008.
Rachmadtullah R, Ms Z, Sumantri MS. Development of computer-based interactive multimedia : study on learning in elementary education. Int J Eng Technol. 2018;7(4):2051–4.
Quadri NN, Qureshi MN, Alsayed AO, Sanober S, Naveed Q.N., Rafik M, et al. Prioritizing barriers of E-Learning for effective teaching-learning using fuzzy analytic hierarchy process (FAHP) E-Learning CSFs and Barriers View project journal View project Prioritizing Barriers of E-Learning for Effective Teaching-Learning using Fuzzy Analytic Hierarchy Process (FAHP). https://www.researchgate.net/publication/322881778. Accessed 4 Dec 2019.
This article resulted from a master's thesis research project (decree code: 1398.819) at Shiraz University of Medical Sciences. We are grateful to the Shiraz University of Medical Sciences (SUMS) research counselor for supporting this study and the Clinical Research Development Centre of SUMS for the statistical analysis. We also wish to thank all faculty members for their honest cooperation and help in conducting the present study.
The authors would like to thank the Vice-chancellor of Research at Shiraz University of Medical Sciences, Shiraz, Iran, for their financial support (grants No. 819).
Ethics approval and consent to participate
The study was approved by the local ethics council of Shiraz University of Medical Sciences (decree code: IR.SUMS.REC.1398.819). First, the study objectives were described to the participants, and their signed informed permission was acquired. To protect anonymity, no names were put on the surveys, and completed questionnaires were decoded by a research assistant to minimize any mistakes. The participants were promised confidentiality of data and that only broad statistics would be published. To respect ethical norms, the participants were guaranteed their free involvement and withdrawal at any moment.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The questionnaire for evaluating the e-flipped learning approach on the knowledge, attitudes, and perceived behaviour of medical educators.
The relationship between Participants' demographic and their knowledge, attitudes, satisfaction, and perceived behaviours.
About this article
Cite this article
Kian, L., Zarifsanaiey, N. & karimian, Z. Effect of the e-flipped learning approach on the knowledge, attitudes, and perceived behaviour of medical educators. BMC Res Notes 15, 227 (2022). https://doi.org/10.1186/s13104-022-06119-8
- Perceived behaviour
- e-flipped learning
- Educational intervention
- Kirkpatrick evaluation model