Skip to main content

On-line virtual patient learning: a pilot study of a new modality in antimicrobial stewardship education for pediatric residents

Abstract

Objectives

Our objective was to develop and validate a virtual patient (VP) learning module to educate pediatric residents about antimicrobial stewardship (AMS) principles. A VP module on complicated pneumonia was developed by experts in AMS and pediatric infectious diseases using the online platform DecisionSim. Decision points were based on AMS principles (diagnosis, antimicrobial selection, dosing, de-escalation, route, duration). Pediatric residents in all training levels at a tertiary pediatric hospital were recruited to test the VP module. Knowledge was assessed via a multiple choice questionnaire. Mean knowledge scores were compared before, after, and 4 months after completing the module using Generalized Linear Mixed Repeated Measures (RM) Analysis. Resident satisfaction was assessed using a validated questionnaire.

Results

Seven of 24 pediatric residents (Years 1–4) completed the VP module and pre- and post-module questionnaires. Mean knowledge scores before, immediately after and 4 months after the module were 58.2%, 66.6%, and 71.6%, respectively. The change in knowledge across time was significant (p < 0.001). Residents were satisfied with the module as an AMS learning strategy.

Introduction

Antimicrobial Stewardship Programs (ASP) aim to measure and improve the appropriate use of antimicrobials in order to optimize clinical outcomes and minimize the emergence of antimicrobial resistance (AMR) [1]. Prescriber education is a recommended antimicrobial stewardship (AMS) strategy to increase knowledge about antimicrobial prescribing and AMR, and to stimulate behavioral change in antimicrobial prescribing [1]. Teaching strategies in AMS education include passive and active approaches [1]. The Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA) recommend incorporating active educational strategies, such as prescriber audit feedback, that are more likely than passive approaches to result in practice changes. [1, 2].

Online learning is one of the suggested methods to actively educate healthcare workers about appropriate antibiotic prescribing, especially in resource limited settings [3, 4]. One online instruction design is Virtual Patient (VP) learning which is defined as “a program that simulates real life clinical scenarios; learners emulate the roles of health care providers to obtain a history, conduct a physical exam, and make diagnostic and therapeutic decisions” [5]. Computerized VP design can be classified into three categories, based the progression of the case: (1) free navigation where the patient’s status remains essentially unchanged as the learner gathers information; (2) linear where the patient’s status evolves over time, but follows the same course regardless of the learner’s decisions; and (3) branching where the patient’s status evolves based on the learner’s decisions [6].VP modules have been used mainly in undergraduate medical education to enhance a variety of outcomes including clinical reasoning and knowledge for various medical conditions [6].

Appropriate antimicrobial prescribing requires good clinical reasoning skills when the clinician collects data, processes the information, develops a differential diagnosis, develops and implements an initial management plan (e.g. decide to prescribe antibiotics, select appropriate antibiotics), and develops a final management plan (e.g. discontinue/modify antibiotics based on patient status and culture results) [7].

Postgraduate trainees play a critical role in shaping antimicrobial utilization patterns in hospitals as part of the front-line clinical team that initiates antimicrobials. However, several studies have reported deviation of prescribing from guidelines for common pediatric infections among pediatric residents [8, 9]. In addition, several studies have shown that the most pediatric residents do not have enough education about AMS during their training [8, 10].

The objectives were to (1) design and validate a VP module to teach pediatric residents the principles of AMS, (2) evaluate the effect of a VP module on pediatric residents’ knowledge about antimicrobial prescribing and AMR and (3) assess residents’ satisfaction with the VP module as an AMS educational strategy.

Main text

Methods

This study included two phases: phase 1 involved the development and validation of the VP module, and phase 2 involved: (a) evaluation of the effectiveness of the VP module at increasing pediatric residents’ knowledge with respect to antimicrobial prescribing and AMR; and (b) evaluation of residents’ satisfaction with the VP module as an educational strategy for AMS education. This study was approved by the Research Ethics Board at the IWK Health Centre, Halifax, Canada. Participants provided written informed consent.

Phase 1: Virtual patient (VP) module development and validation

The authors designed the VP module using the online platform DecisionSimTM, Version 3.2.2, (Kynectiv, Inc., Chadds Ford, PA, USA). A pediatric complicated community acquired pneumonia (CCAP) case was developed using branched design. The six main decision points were based on AMS principles: optimizing diagnosis, antimicrobial selection, dosing, de-escalation, administration route, and duration of treatment [2]. National and international guidelines for management of CCAP as well as local susceptibility patterns for common bacteria (antibiogram) were incorporated into the case. Factors that promote AMR, including inappropriate antimicrobial use, were also presented at the end of the case using text and video clips. Learners received feedback on each decision they made.

The module was evaluated for content validity by 3 infectious diseases physicians and 1 medical microbiologist using a published reviewer checklist for VP design [11]. This questionnaire consisted of 32 questions using a 5 point Likert scale (from strongly disagree to strongly agree) covering authenticity of the patient encounter and consultation, professional approach in the consultation, coaching during consultation, and overall judgment of the case, as well as open-ended questions about strengths and weaknesses. The appropriateness of the case and ease of navigation were evaluated by three pediatric residents from the University of Toronto, Ontario, Canada using a previously published validated online survey [12]. This survey consisted of 12 questions using a 5 point Likert scale (from strongly disagree to strongly agree) covering authenticity of the VP case, engagement with the case, coaching during the case, learning effectiveness of the case (i.e. preparedness to manage a real life case), overall judgment of the case, as well as open-ended questions about strengths and weaknesses [12].

Phase 2a: Evaluation of the effectiveness of the VP module

This phase was designed as a pre- and post-intervention evaluation to assess the effect of the VP module on pediatric residents’ knowledge about antimicrobial prescribing and AMR. Potential participants were recruited through email invitation with two follow up email reminders from January to February 2016. A consent form was included in the email invitation. Eligible participants included pediatric residents at all training levels (Postgraduate Year 1 to 4) (N = 24) at the IWK Health Centre.

A previously published validated survey [10], modified to reflect our center’s local antimicrobial susceptibility patterns, was used to measure residents’ mean knowledge scores about AMS principles at three time points: Time point 1: before completing the VP module, Time point 2: immediately after completing the module, and Time point 3: four months after completing the module. The survey included seven “single best answer” multiple choices questions (MCQs). Four MCQs assessed knowledge of AMR and knowledge of local antimicrobial susceptibility patterns of the most common bacteria in pediatric infections; and three MCQs covered application of antimicrobial prescribing principles in pediatric hospital settings (Additional file 1: knowledge MCQs). After completing the pre-test, participants were given access to the online VP module to complete within 1 week. Post-test surveys were sent to participants via email after completing the module. Three MCQs that covered application of antimicrobial prescribing in pediatric hospital settings were changed in the second post-test.

Phase 2b: Evaluation of residents’ satisfaction with the VP module

Residents’ satisfaction with the VP modules for antimicrobial education was assessed using the same questionnaire used in the validation phase [12], following completion of the first post-test survey.

Statistical analysis

The mean and standard deviation were used to describe knowledge score. Missing scores from post-test (N = 2/9) were replaced by the mean. We transposed the data from wide to long data using the data restructure feature resulting in 9 physicians times 3 repeated measures which equaled 27 data records. Then we conducted Generalized Linear Mixed Repeated Measures (RM) Analysis to assess the effect of the VP module on participants’ knowledge across time. Time point 3 (4 months after completing the module) was used as reference for comparison in the analysis. Independent groups t test was used to compare mean knowledge score between postgraduate year (PGY) 1 or 2 residents and PGY 3 or 4 residents. Survey responses regarding residents’ satisfaction with the module were reported as proportions.

Results

Virtual patient (VP) module development and validation

All 4 faculty reviewers agreed that the case represented a typical clinical scenario. Three of 4 reviewers agreed that the case triggered the learners’ clinical reasoning. One reviewer felt that the case encouraged over investigation (Additional file 2: Reviewer checklist). The case was edited based on reviewers’ comments. All 3 residents who piloted the module agreed that it was easy to navigate and reflected a real life case. The key features of the VP module are shown in Table 1.

Table 1 Key Features of a Virtual Patient module to educate pediatric residents about principles Antimicrobial Stewardship Practices

Effectiveness of the VP module on pediatric residents’ knowledge about antimicrobial prescribing and AMR

Nine of 24 pediatric residents completed the pre-test knowledge survey, of which seven completed the VP module and post intervention knowledge questionnaires. The response rate for completing the pre and post-test phases was 29% (7/24). Of these 7 residents: 3 were in PGY 1–2 while 4 were in PGY 3–4. Overall mean knowledge scores before, immediately after, and 4 months after the module were 58.2%, 66.6%, and 71.6%, respectively. Knowledge scores were significantly lower at time point 1 than time point 3 (p = 0.003), while scores at time points 2 and 3 were similar (p = 0.33) (Table 2).

Table 2 Generalized linear repeated measures analysis of the effect of the VP module on participants’ knowledge score across time (N = 27)

Mean knowledge score by level of training is shown in the Additional file 3: Table S1. There was no significant difference in pre-test knowledge scores between PGY 1–2 and PGY 3–4, p = 0.25.

Participants’ satisfaction with the VP modules for AMS education

Seven participants completed the VP module and the satisfaction survey. Satisfaction with the VP module was high overall (i.e. all agreed or strongly agreed) in terms of its simulation of real life, engagement with the case, feedback on the participant’s decisions, and perceived preparedness for managing a similar real life case (Table 3).

Table 3 Residents’ Satisfaction Survey (N = 7)

Discussion

The development and validation phase showed that the online VP module simulated real life cases and enhanced learning of AMS principles and AMR among pediatric residents. VP modules may also enhance clinical reasoning skills in antimicrobial prescribing as shown in the validation phase. Pediatric residents’ knowledge about principles of antimicrobial prescribing and AMR appeared to increase after completing a VP module that incorporates AMS principles.

Based on our literature review, this is one of first studies to assess the effectiveness of online VP modules in postgraduate AMS education. Most prior studies reporting on AMS instructional methods assessed passive learning methods (e.g. group presentations and guideline dissemination) [13, 14]. Such methods are unlikely to result in sustained behavioral improvement in antimicrobial prescribing [2].

VP modules have been used mainly in undergraduate medical education and there is growing interest in post-graduate teaching [6]. Cook et al. has published a systemic review and meta-analysis on the use of computerized VP modules in health professions education that showed improvement in knowledge and clinical reasoning skills when VP is compared with no intervention or non-computer instruction. However there was considerable heterogeneity in study design and populations included in these studies [6]. A recent study evaluated the use of online learning in AMS education among 606 medical students. Six online modules were developed covering different AMS topics. The modules included both theoretical information as well as virtual patients. Significant improvement in mean knowledge scores was observed between pre and post intervention assessments (5.78 vs 8.45, p < 0.001) [15]. Similarly, Heath et al. reported an improvement in AMS knowledge among 103 nurses at long-term care facilities after completing online learning modules. Pre and post knowledge scores were 75% and 86%, respectively (p < 0.001). However, details of the instructional methods used were not described [16].

Resident physicians expressed a high level of satisfaction with the VP for AMS education. All participants agreed that the module simulated real-life cases that they might encounter in their daily practice. They also reported that it stimulated their clinical reasoning skills and prepared them to manage a similar case. These findings are in agreement with most studies that evaluated online VP modules [6].

In conclusion, VP modules present an innovate approach to AMS education among resident physicians that merits further study.

Limitations

This study was a pilot research project and was limited to one module and a small sample of participants, which limits the generalizability of the findings. In addition, we could not incorporate other features in the VP module such as tracking respondents’ time due to limited funding. We were also unable to measure how completion of the VP module affected residents’ antimicrobial prescribing behavior.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its additional information files]. Figure of example of branching node in the VP module is available upon request from the corresponding author.

Abbreviations

ASP:

Antimicrobial Stewardship Programs

AMR:

Antimicrobial resistance

VP:

Virtual patient

CAP:

Community acquired pneumonia

CCAP:

Complicated community acquired pneumonia

PGY:

Postgraduate year

References

  1. 1.

    Ohl C, Luther V. Health care provider education as a tool to enhance antibiotic stewardship practices. Infect Dis Clin. 2014;28(2):177–93.

    Article  Google Scholar 

  2. 2.

    Barlam T, Cosgrove S, Abbo L, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016;62(10):e51–77.

    Article  Google Scholar 

  3. 3.

    Hara GL. Antimicrobial stewardship in hospitals: does it work and can we do it? J Glob Antimicrobial Resist. 2014;2(1):1–6.

    Article  Google Scholar 

  4. 4.

    Goff D, Kullar R, Bauer K, File TM Jr. Eight habits of highly effective antimicrobial stewardship programs to meet the joint commission standards for hospitals. Clin Infect Dis. 2017;64(8):1134–9.

    Article  Google Scholar 

  5. 5.

    Association of American Medical Colleges. Effective use of educational technology in medical education: summary report of the 2006 aamc colloquium on educational technology. Washington, DC: AAMC Institute for Improving Medical Education; 2007.

    Google Scholar 

  6. 6.

    Cook D, Erwin P, Triola M. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010;85(10):1589–602.

    Article  Google Scholar 

  7. 7.

    Amey L, Donald K, Teodorczuk A. Teaching clinical reasoning to medical students. Br J Hosp Med. 2017;78(7):399–401.

    Article  Google Scholar 

  8. 8.

    Shukla P, Behnam-Terneus M, Cunill-De Sautu B, Perez G. Antibiotic use by pediatric residents: identifying opportunities and strategies for antimicrobial stewardship. Hosp Pediatr. 2017;7(9):553–8.

    Article  Google Scholar 

  9. 9.

    Fakih M, Hilu R, Savoy-Moore R, Saravolatz L. Do resident physicians use antibiotics appropriately in treating upper respiratory infections? A survey of 11 programs. Clin Infect Dis. 2003;37(6):853–6.

    Article  Google Scholar 

  10. 10.

    Bowes J, Yasseen AS III, Barrowman N, et al. Antimicrobial stewardship in pediatrics: focusing on the challenges clinicians face. BMC Pediatr. 2014;14(1):212.

    Article  Google Scholar 

  11. 11.

    De Leng B, Huwendiek S, Donkers J, and eViP project team. Reviewer checklist for virtual patient design. Retrieved from Electronic virtual patients (eViP): https://virtualpatients.eu/resources/evaluation-tool-kit/. Accessed 29 Sept 2015.

  12. 12.

    Huwendiek S, De Leng BA, Kononowicz AA, et al. Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning. Med Teach. 2015;37(8):775–82.

    Article  Google Scholar 

  13. 13.

    Roque F, Herdeiro MT, Soares S, Rodrigues AT, Breitenfeld L, Figueiras A. Educational interventions to improve prescription and dispensing of antibiotics: a systematic review. BMC Public Health. 2014;14(1):1276.

    Article  Google Scholar 

  14. 14.

    Rocha-Pereira N, Lafferty N, Nathwani D. Educating healthcare professionals in antimicrobial stewardship: can online-learning solutions help? J Antimicrob Chemother. 2015;70(12):3175–7.

    CAS  PubMed  Google Scholar 

  15. 15.

    Laks M, Guerra CM, Miraglia JL, Medeiros EA. Distance learning in antimicrobial stewardship: innovation in medical education. BMC Med Educ. 2019;19(1):191.

    Article  Google Scholar 

  16. 16.

    Heath B, Bernhardt J, Michalski TJ, et al. Results of a Veterans Affairs employee education program on antimicrobial stewardship for older adults. Am J Infect Control. 2016;44(3):349–51.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge Dr. Joanne Langley, Dr. Scott Halperin (Dalhousie University), Dr. Sarah Khan (McMaster University), and Dr. Khalifa BinKhamis (King Saud University, Riyadh, Saudi Arabia) for their contributions to the content validation of the VP module.

Funding

The Corresponding author (AA) has received educational grant (discount) from Kynectiv, Inc., Chadds Ford, PA, USA) (formal name: DecisionSim™) for the VP platform for learners’ licenses.

Author information

Affiliations

Authors

Contributions

Primary investigator (AA) wrote the study protocol, the online VP case and prepared the surveys and the manuscript draft. Co-author (KT) was the supervisor and supervised the whole project. KS and EB revised the on-line module and revised the manuscript. All authors read and approved the manuscript.

Corresponding author

Correspondence to Amer Alshengeti.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Research Ethics Board at the IWK Health Centre, Halifax, Canada, REB #1019960. Participants provided written informed consent.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Antimicrobial Stewardship (AMS) Knowledge Questions.

Additional file 2.

Reviewer checklist.

Additional file 3.

Descriptive statistics of residents’ knowledge score Pre- and Post VP module.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alshengeti, A., Slayter, K., Black, E. et al. On-line virtual patient learning: a pilot study of a new modality in antimicrobial stewardship education for pediatric residents. BMC Res Notes 13, 339 (2020). https://doi.org/10.1186/s13104-020-05170-7

Download citation

Keywords

  • Virtual patient
  • Antimicrobial stewardship
  • Antimicrobial resistance
  • Antibiotics education