- Research note
- Open Access
A simulated patient evaluation of pharmacist’s performance in a men’s mental health program
BMC Research Notesvolume 11, Article number: 765 (2018)
The Headstrong program, a pharmacy based men’s mental health promotion program, was designed to enhance pharmacists’ care of men with mental illness and addictions and was focused on six conditions. A simulated patient (SP) encounter on insomnia was used to evaluate pharmacist’s performance as a part of the Headstrong program.
Six Headstrong pharmacists consented to participate in the SP encounter as part of the evaluation of the Headstrong program. Pharmacists’ mean scores in most categories that were evaluated (e.g., pre-supply/assessment score, sleep score) were lower than expected. In assessing the SP during the encounter, pharmacists’ mean score was 5.7 (SD 2.0) of a possible 13 points. No pharmacists asked about the SP’s age, availability of other supports, allergies, and whether they had an existing relationship with a pharmacist. One pharmacist inquired about medical conditions, and two asked about pre-existing mental health conditions. Three pharmacists inquired about concurrent medications. The Headstrong program was discussed by half of the pharmacists and a resource recommended by the Headstrong program was suggested by one pharmacist. Several pharmacists used self-disclosure as a mechanism to support rapport building. Overall, the SP felt cared for and respected by the pharmacists and had confidence in their knowledge.
Headstrong-Taking Things Head On, hereafter referred to as Headstrong, is a program targeting men in Nova Scotia, Canada, living with mental illness and addictions. The program was launched in Nova Scotia in 23 pharmacies in October, 2017 and was offered in pharmacies for a 6 month period. Headstrong has six areas of focus: suicide, alcohol use, tobacco use, problems with sleep, depression, and anxiety. Headstrong included an education and training program for pharmacists, including a live, face-to-face training session. The pharmacists were also enrolled in an online course, which included modules for the six content areas in addition to modules on motivational interviewing. Pharmacists were provided with signage for their pharmacies to alert existing patients and other customers that pharmacists are a resource that can help with men’s mental health. A website, headstrong.life, was also developed with curated resources including books, apps, and other recommended websites that pharmacists could use and recommend.
A simulated patient (SP) exercise was proposed a priori as part of the Headstrong evaluation following the education and training. Simulated patients have been used as a feasible and useful method for assessing various pharmacist behaviours (e.g., communication skills) and for improving pharmacy personnel’s knowledge and perceptions of their abilities [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]. We report on the results of SP encounters as a part of the Headstrong program evaluation.
We modelled the simulated patient on the work of Kippist et al.  and our previous experience . The scenario included a male patient with acute insomnia in their mid-twenties who was a full-time student struggling with insomnia due to final exams. The SP was seeking advice and resources to help overcome their insomnia, with underlying issues of school-related stress and anxiety. One male SP was used for all of the interactions. The SP was instructed to use a standard approach with each pharmacist.
Several categories were evaluated numerically and summarized using descriptive statistics (Table 1). There were additional sections for qualitative comments in the categories of: overall experience, care received, knowledge of the pharmacist, addressing needs, and interaction with other staff. Pharmacists could also provide feedback to the SP after the encounter.
The SP was hired through Centre for Collaborative Clinical Learning and Research (C3LR) from Dalhousie University .
The Headstrong pharmacists from the 23 Headstrong pharmacies were not required to participate in the SP exercise as part of their participation in the Headstrong program. To meet ethical requirements, consent was required from the pharmacists for the SP activity. All Headstrong program pharmacists were approached by the research coordinator to request participation in the SP activity. Once a pharmacist consented, their schedule for several months in advance and a photo, for easy recognition by the SP, were requested. Pharmacists were not aware of the SP’s visit details (e.g., date and time), appearance, or chief complaint. SP visits occurred at any time between January and March of 2018 and consenting pharmacists were visited once.
For each visit, the SP interacted with pharmacist and then left the pharmacy to complete scoring. The SP then returned that same day to disclose that the deception had occurred and offered to review the information with the pharmacist immediately or at a later time. No further follow-ups with the SP occurred.
Six pharmacists consented to participating. Following the encounter, all pharmacists agreed to receive and provide feedback immediately.
Pharmacists scored an average of 5.7 (SD 2.0) (Table 2). There were four of 13 variables in which no pharmacists scored points: identifying the patients age, identifying the patients supports, inquiring about any allergies, and inquiring about the patient’s relationship with pharmacists. Categories covered by at least one but fewer than four pharmacists included medications (n = 3), medical conditions (n = 1), mental health conditions (n = 2), and substance use (n = 4).
Five of six pharmacists inquired about the duration of the problem, while all six pharmacists inquired about the patient’s symptoms, the recurrence of the problem, and previous treatments for insomnia.
Headstrong specific scores
Of the six pharmacists, three scored four or above (mean 2.2; SD 2.4) (Table 2) points and the remaining achieved zero for Headstrong criteria. Three of the six pharmacists discussed and explained Headstrong and showed or referred the SP to the website. One pharmacist recommended at least one Headstrong resource and two provided them with Headstrong print materials. In general comments, the SP commented that Headstrong promotional materials were at the counter in two stores.
The mean sleep score was 6 (SD 1.9) (Table 2). None of the pharmacists inquired about sleep apnea but all inquired about potential causes of acute insomnia. The same five of six pharmacists inquired about difficulty initiating, difficulty maintaining sleep, and principles of sleep hygiene/non-medication resources. Two pharmacists provided information on sleeping and insomnia.
All but one pharmacist provided guidance on sleep hygiene and three promoted specific non- medication resources for sleep.
All pharmacists identified the medication name during medication recommendations, and informed the SP when they should take the medication. Five of six outlined the dosage and duration of use. One pharmacist discussed the cost of the medication.
The highest score for communication effectiveness was 75 out of a possible 77 (Table 2). The category with the lowest mean (4.2; SD 2.0) was inquiring about the need for clarification for any advice or recommendations. All pharmacists scored a maximum of 7 points regarding being respectful.
Overall rating of quality
The overall quality ratings ranged from six to 10 (Table 2).
Qualitative comments by the SP
Overall experience in the pharmacy
The SP noted positive experiences overall with pharmacists, feeling that there was an eagerness to help and support them with their sleep problem. They felt comfortable and cared for during the interactions. In two interactions, the SP reported that the availability of Headstrong print materials at the pharmacy counter helped them to initiate the conversation with the pharmacist. The SP noted that in two of the encounters, there were no other customers nearby and in all others, there were one or more customers in the pharmacy area.
Level of care received
The SP reported that several pharmacists directed the SP to an aisle with over the counter medication options for sleep to discuss the nature and duration of their sleep problem, triggers, and to suggest solutions. Pharmacists recommended various options, including over-the-counter medication, sleep hygiene practices, and Headstrong materials. In one instance, the pharmacist emphasized the importance of connecting with their family doctor if their sleep issues continued, which made the SP feel cared for and supported. Some pharmacists took the time to understand the underlying causes of the SP’s sleeping problems, and suggested solutions with their unique circumstances as a student in mind.
Pharmacists’ level of knowledge
The SP felt confident in the knowledge of each pharmacist. Pharmacists often kept the patient’s circumstances at the forefront of their discussion and recommendations, to ensure the solutions being presented would not negatively affect the SP’s schooling. On some occasions, the SP noted the interactions could have been improved by the pharmacists “digging deeper” to learn more about the patients’ history with sleep problems and/or high levels of stress, and the overall duration of the problem. Resources and information provided to the SP included sleep efficiency information, sleep hygiene practices, a hard-copy sleep journal, and more than one over-the-counter medication option, both for short-term and long-term solutions.
Addressing patient needs
The SP felt that their sleep needs were addressed in all interactions. The SP commented on the appreciation for the empathy expressed by pharmacists. In some cases, the SP indicated that they would have preferred more exploration in terms of non-medicinal solutions, such as sleep hygiene. In addition, the SP hoped that more pharmacists would have taken the time to explore the underlying cause of insomnia (i.e., stress).
Additional comments by the simulated patient
The SP valued pharmacists’ techniques to build rapport including humour, self-disclosure, and making an effort to learn more about them (e.g., inquiring regarding area of study). These efforts made the SP feel more comfortable. The SP noted appreciation for the discussion of costs on a student’s budget. For example, one pharmacist noted the higher cost of some subscription-based cognitive behavior therapy for insomnia resources on the Headstrong website.
Participating pharmacists’ comments
Pharmacists thought the SP case flowed naturally. One pharmacist reported, “The scenario was believable, [and] something I would encounter in my day-to-day practice” (pharmacist 2). Pharmacist 3 also reported that it “wasn’t my best work” and explained their unique circumstances that hindered performance. This included issues of short-staffing, which focused the pharmacist’s time and efforts on preventing prescription-related errors in the dispensary. It was, however, reported as a valuable learning opportunity with appreciation of the feedback. Another pharmacist appreciated the feedback on their lack of emphasis on non-medicinal resources and supports for insomnia.
Using SP encounters to measure pharmacists’ performance in a men’s mental health promotion program was useful to identify potential areas of improvement for pharmacists regarding their assessment and management of acute insomnia. More than half of the pharmacists promoted nonpharmacological supports for insomnia including, but not limited to, Headstrong resources, which was emphasized during the Headstrong education and training. Some areas such as inquiring about concurrent medical conditions, medications, mental health conditions, and allergies were not assessed as would have been expected. This finding concurs with existing literature. In a study on pharmacists’ recommendations in Jordan for over the counter medications for headache, medication recommendations were frequently made without obtaining essential information about the SP’s symptoms or medical history . This also occurred in a community pharmacy SP study in Australia with smoking cessation . In a previous telephone-based insomnia SP study, 35% of the intervention versus 10% of control group pharmacists inquired about medical conditions . Our findings in context with the broader literature in this area suggest that pharmacists assessing patients without adequate knowledge of their medical conditions may lead to harmful recommendations. This could occur through several mechanisms. For example, failing to rule out whether an underlying condition could be exacerbating or causing the presenting illness may lead to unnecessary use of a medication, which inherently may have adverse effects. Recommending a medication without knowledge of underlying disease processes may also put the patient at risk for harm if a medication were to worsen symptoms of the disease.
In our study, all pharmacists asked regarding previous treatments for insomnia, yet half inquired about concurrent medication use. Issues with pharmacists acquiring incomplete medication histories have been reported by others. A study of SPs obtaining prescription medications for diabetes and asthma in Qatar demonstrated that fewer than 4% of pharmacists inquired regarding concurrent medications . Similarly in the smoking cessation study in Australia by Saba et al. , fewer than 5% of the simulations with a pregnant woman were asked about concurrent medications. For the second scenario regarding an older adult with cardiovascular disease who was attempting to quit smoking, fewer than one in four of the encounters included questioning about concurrent medications . In the telephone simulation regarding insomnia, more pharmacists in the intervention group (62%) compared to the controls (48%) asked the SP regarding their concurrent medications . Although 69% of SPs with gastroesophageal reflux symptoms presenting to community pharmacists in Australia had an adequate medication history taken, three in 10 did not before pharmacists made recommendations . These findings have important ramifications given the potential for pharmacodynamic and pharmacokinetic drug–drug interactions that can occur when combining treatments.
For medication recommendations, all pharmacists in our study discussed the name and when to take it, but not all discussed the dose, the expected onset of effect, how long to use it, side effects, and only one pharmacist discussed cost. Saba et al. similarly reported that the majority of encounters for smoking cessation products lacked information given by the pharmacist such as when to take the medication, adverse effects, duration of therapy, and how to use the products . These findings are important from a shared decision making perspective as many patients prefer and value getting this type of information (e.g., duration of treatment) to inform their treatment choices .
Six pharmacists from the 23 participating pharmacies in the Headstrong program consented to participation in the SP activity.
Pharmacists may have changed their behaviour knowing that a SP visit was possible during the 3 month observation period.
There was no control group and the pharmacists were not randomized to receive an SP visit.
We used existing literature and our own experience to develop the standardized patient case and assessments, which were not validated.
The SP encounters were not video or audio recorded. The data analysis was conducted on information transcribed from what the SP recorded immediately following the encounters.
Saba M, Diep J, Bittoun R, Saini B. Provision of smoking cessation services in Australian community pharmacies: a simulated patient study. Int J Clin Pharm. 2014;36:604–14.
Dolovich L, Sabharwal M, Agro K, Foster G, Lee A, McCarthy L, Willan AR. The effect of pharmacist education on asthma treatment plans for simulated patients. Pharm World Sci. 2007;29:228–39.
Weiss MC, Booth A, Jones B, Ramjeet S, Wong E. Use of simulated patients to assess the clinical and communication skills of community pharmacists. Pharm World Sci. 2010;32:353–61.
Paravattil B, Kheir N, Yousif A. Utilization of simulated patients to assess diabetes and asthma counseling practices among community pharmacists in Qatar. Int J Clin Pharm. 2017;39:759–68.
Watson MC, Skelton JR, Bond CM, Croft P, Wiskin CM, Grimshaw JM, Mollison J. Simulated patients in the community pharmacy setting. Using simulated patients to measure practice in the community pharmacy setting. Pharm World Sci. 2004;26(1):32–7.
MacFarlane B, Matthews A, Bergin J. Non-prescription treatment of NSAID induced GORD by Australian pharmacies: a national simulated patient study. Int J Clin Pharm. 2015;37:851–6.
Bray BS, Schwartz CR, Odegard PS, Hammer DP, Seybert AL. Assessment of human patient simulation-based learning. Am J Pharm Educ. 2011;75:208.
Lin K, Travlos DV, Wadelin JW, Vlasses PH. Simulation and introductory pharmacy practice experiences. Am J Pharm Educ. 2011;75:209.
Kane-Gill S, Smithburger PL. Transitioning knowledge gained from simulation to pharmacy practice. Am J Pharm Educ. 2011;75:210.
Collier IA, Baker DM. Creation of an active learning healthcare communications course using simulations relevant to pharmacy practice. Curr Pharm Teach Learn. 2017;9:626–32.
Collins JC, Schneider CR, Naughtin CL, Wilson F, de Almeida Neto AC, Moles RJ. Mystery shopping and coaching as a form of audit and feedback to improve community pharmacy management of non-prescription medicine requests: an intervention study. BMJ Open. 2017;7:e019462.
Seubert LJ, Whitelaw K, Hattingh L, Watson MC, Clifford RM. Interventions to enhance effective communication during over-the-counter consultations in the community pharmacy setting: a systematic review. Res Social Adm Pharm. 2017. https://doi.org/10.1016/j.sapharm.2017.12.001.
Dabaghzadeh F, Hajjari R. Practice of community pharmacists related to multivitamin supplements: a simulated patient study in Iran. Int J Clin Pharm. 2018;40:190–5.
Lynch SE, Griffin BL, Vest KM. Assessment of a simulated contraceptive prescribing activity for pharmacy students. Curr Pharm Teach Learn. 2018;10:178–84.
Al Aqeel S, Abanmy N, AlShaya H, Almeshari A. Interventions for improving pharmacist-led patient counselling in the community setting: a systematic review. Syst Rev. 2018;7:71.
Collins JC, Schneider CR, Moles RJ. Emergency contraception supply in Australian pharmacies after the introduction of ulipristal acetate: a mystery shopping mixed-methods study. Contraception. 2018;98:243–6.
Zolezzi M, Abdallah O, Kheir N, Abdelsalam AG. Evaluation of community pharmacists’ preparedness for the provision of cardiovascular disease risk assessment and management services: a study with simulated patients in Qatar. Res Soc Adm Pharm. 2018. https://doi.org/10.1016/j.sapharm.2018.04.032.
Hammad EA, Elayeh E, Tubeileh R, Watson M, Wazaify M. A simulated patient study assessing over the counter supply and counseling in Jordan: responding to headache complaints. Int J Clin Pharm. 2018. https://doi.org/10.1007/s11096-018-0679-8.
Murphy A, Martin-Misener R, Kutcher S, Gardner D. Pharmacists’ performance in a telephone-based simulated patient study after a mental health capacity-building program. Int J Clin Pharm. 2015;37:1009–13.
Kippist C, Wong K, Bartlett D, Saini B. How do pharmacists respond to complaints of acute insomnia? A simulated patient study. Int J Clin Pharm. 2011;33:237–45.
Centre for Collaborative Clinical Learning and Research, Dalhousie University. 2018. https://www.dal.ca/faculty/interprofessional-education/facilities/centre.html. Accessed 3 July 2018.
Young A, Tordoff J, Smith A. ‘What do patients want?’ Tailoring medicines information to meet patients’ needs. Res Soc Adm Pharm. 2017;13:1186–90.
AM and DG conceptualized the project, developed the simulated patient case, and drafted the manuscript. Both authors read and approved the final manuscript.
The authors thank: Taylor Currie, who was hired as a project manager, for compiling the reports from the simulated patient; Drs. Stan Kutcher and Ruth Martin-Misener for their contributions to the successfully funded application; and Ms. Sophie Peltekian for technical support in preparing the manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
Individual pharmacists’ raw data is not available as consent was not obtained from the pharmacists for this purpose.
Consent of publication
Ethical approval and consent to participate
Dalhousie University Research Ethics Board File Number 2015-3728. Written informed consent was required and obtained for participants.
The Headstrong-Taking Things Head-On project is funded by the Movember Foundation. The funder did not have input regarding the design, data collection, analysis, interpretation, or writing of the simulated patient study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.