Faculty perceptions and practice impact of a faculty development workshop on emergency medicine teaching

Faculty Development

Glen Bandiera, MD, MEd;* Shirley Lee, MD, MHSc;† John Foote, MD‡
*Assistant Professor, Clinician Educator, Department of Medicine, University of Toronto, Toronto, Ont.; Staff Emergency Physician, Trauma Team Leader, St. Michael's Hospital, Toronto, Ont. †Assistant Professor, Department of Family and Community Medicine, University of Toronto; Emergency Physician, Assistant Director, Schwartz-Reisman Emergency Department, Mt. Sinai Hospital, Toronto, Ont. ‡Assistant Professor, Department of Family and Community Medicine, University of Toronto; Emergency Physician, Credit Valley Hospital, Mississauga, Ont.

CJEM 2005;7(5):321-327

Abstract

Background: Faculty development initiatives to improve emergency department (ED) teaching are compromised by the paucity of information about what behaviours and characteristics are desirable in an emergency medicine (EM) teacher.

Objectives: To design and evaluate a learner-centred, interactive faculty development workshop based on original ED teaching research.

Methods: Registrants for a university-based faculty development workshop on ED teaching completed a needs assessment and pre-workshop self-reflection exercise. Responses were grouped into 3 themes derived from the ambulatory teaching literature and a recent survey of expert clinical EM teachers and learners. Participants underwent a half-day workshop consisting of 1 large group interactive session and 3 small group sessions using role playing, practice reflection, real time review of hard copy resources, and brainstorming. Evaluation included a post-event ordinal scale questionnaire and a 4-month follow-up short answer survey, both measuring participants' perceptions of workshop effectiveness.

Results: Fifteen faculty participated. The needs assessment identified "Common mistakes," "Teaching efficiently" and "Dealing with the difficult learner" as themes. All 15 completed evaluations, rating the workshop as relevant (4.6/5), specific to their needs (4.4/5) and useful (4.5/5). At 4 months, 10 out of 10 respondents reported success at implementing new techniques and 8 reported greater confidence in teaching. The most common new techniques were: setting better learning objectives, giving better feedback, actively seeking teaching opportunities, and identifying a teaching point.

Conclusions: Learner-centred faculty development meets perceived needs of faculty and can result in participants trying new teaching strategies.

Résumé

Contexte : La rareté de l'information sur les comportements et les caractéristiques souhaitables chez un formateur en médecine d'urgence (MU) compromet les initiatives de perfectionnement des enseignants qui visent à améliorer l'enseignement aux services d'urgence.

Objectifs : Concevoir et évaluer un atelier interactif de perfectionnement des enseignants axé sur les apprenants et fondé sur une recherche originale relative à l'enseignement au service d'urgence.

Méthodes : Des apprenants inscrits à un atelier universitaire de perfectionnement d'enseignants portant sur la formation au service d'urgence ont participé à un exercice d'évaluation des besoins et d'autoréflexion préalable à l'atelier. On a regroupé les réponses en trois thèmes dérivés des publications sur la formation en services ambulatoires et d'un sondage réalisé récemment auprès de formateurs experts en MU clinique et d'apprenants. Les participants ont suivi un atelier d'une demi-journée comportant une séance interactive pour tout le groupe et trois ateliers au cours desquels on a utilisé la simulation, la réflexion sur la pratique, l'étude en temps réel de documents sur support papier et une séance de remue-méninges. L'évaluation a inclus un questionnaire sur une échelle ordinale après l'événement et un questionnaire à réponse courte de suivi à quatre mois, qui ont tous deux mesuré les perceptions des participants au sujet de l'efficacité de l'atelier.

Résultats : Quinze enseignants ont participé. L'évaluation des besoins a dégagé comme thèmes «Les erreurs communes», «La formation efficiente» et «Le problème de l'apprenant difficile». Les 15 participants ont rempli les évaluations et jugé l'atelier pertinent (4,6/5), spécifique à leurs besoins (4,4/5) et utile (4,5/5). À quatre mois, 10 répondants sur 10 ont signalé avoir réussi à mettre en œuvre de nouvelles techniques et huit ont déclaré avoir davantage confiance dans la formation. Les nouvelles techniques les plus répandus consistaient à fixer de meilleurs objectifs d'apprentissage, fournir de meilleurs commentaires, rechercher activement les possibilités de formation et trouver un point de formation.

Conclusions : Le perfectionnement des enseignants axé sur l'apprenant répond aux besoins perçus des enseignants et peut amener les participants à faire l'essai de nouvelles techniques pédagogiques.

See also: Commentary

Introduction

Teaching in the ambulatory setting is becoming more common, generating increased interest in the best teaching techniques for this environment. To help faculty prepare for this challenge, effective faculty development programs should include teaching improvement initiatives.1 Recent review articles have identified character traits of good clinical teachers and models for effective ambulatory teaching;2-8 however, none of these reviews found original research on teaching in the emergency department (ED) setting. Emergency medicine (EM) faculty developers interested in using educational literature to inform their content development are therefore required to extrapolate and adapt strategies and models derived in other ambulatory settings, which may not be appropriate. Recent research, including surveys of accomplished EM teachers across Canada and focus groups with various levels of EM learners at 5 Ontario medical schools, were used to develop a comprehensive list of strategies and techniques for effective ED teaching.9,10 This new research may inform the content development of teaching instruction workshops tailored specifically to ED practice.

Faculty development programs should maximize learning among participants. Research suggests that successful faculty development initiatives should involve learners in goal-setting, should be interactive, include opportunities for practice, respect a variety of adult learning styles, and be contextual.11-15 Additionally, faculty development initiatives should be assessed for effectiveness using both immediate and delayed assessments.16 We sought to use these principles to conduct and evaluate a learner-centred ED teaching faulty development workshop based on recent research into the practices of recognized EM teachers in Canada.

Methods

A half-day faculty development workshop was advertised in a university-based academic EM centre. Enrolment was capped at 15 to preserve a high faculty-to-participant ratio. A month before the workshop, each registrant completed a needs assessment survey that asked them to list 4 issues they would like to see addressed. In addition, they were asked to reflect on their own teaching behaviours and bring a written assessment for discussion at the workshop.

A previously completed survey of expert EM teachers and learners across Canada was used to help adapt accepted ambulatory teaching models to the ED setting and to compile a comprehensive manual of practical ED teaching strategies.9 Two award-winning faculty were recruited to lead small group workshops, and resident volunteers were solicited to role play difficult learner scenarios. The needs assessment responses were reviewed by a single faculty member and categorized into 3 major themes, each to be the focus of 1 small group workshop.

On the day of the workshop, participants were randomly assigned to 1 of 3 groups. An interactive large group session was conducted to discuss ED teaching challenges, to introduce the themes and to develop goals and objectives for the breakout sessions. In a small group session that focused on difficult trainees, participants took turns using a scripted role-modeling exercise with a trained EM resident and received feedback on their performance. In a second small group session, participants drew from their past experiences to brainstorm a list of common mistakes. This list was compared with one generated by the expert teachers, and discussion ensued regarding similarities and differences. In the third small group session, participants identified specific challenges to efficient teaching and used a comprehensive teaching manual to develop a solution. They then discussed the solutions in the context of the original case. After each small group rotated through all 3 60-minute sessions, the large group reconvened and participants listed their most important lesson from the day and reported what they would incorporate into their teaching practice.

Fig. 1. Post-workshop questionnaire.

UNIVERSITY OF TORONTO - DIVISIONS OF EMERGENCY MEDICINE
CONTINUING MEDICAL EDUCATION WORKSHOP

Effective Teaching Strategies for the Emergency Department

Course Evaluation Form

Please rate today's workshop on each of the numbered dimensions using the following scale:

1 = poor 2 = needs improvement 3 = as expected 4 = better than expected 5 = exemplary
  1.  Pre-workshop notification and communication.
  2.  The relevance of the stated goals of the workshop to your practice.
  3.  Ability of the faculty to tailor the workshop to the needs of the participants.
  4.  Organization of the workshop (clarity, agenda, logistics).
  5.  Amount of time spent in interactive learning.
  6.  Appropriateness of teaching aids (AV, overhead, etc.).
  7.  Knowledgeable faculty.
  8.  Approachable faculty.
  9.  Style of presentation.
  10.  The usefulness of the content.
  11.  The facility.
  12.  Catering.

Yes / NoWould you attend a similar workshop exploring different issues in ED teaching?

Yes / NoWould you recommend this workshop to peers?

Comments/Suggestions?

 
 

Program evaluation was based on participants' self-reported perceptions and performance. Participants used 5-point ordinal scales to rate the program in 10 domains and the setting in 2 domains (Fig. 1). Registrants were contacted by email 4 months after the program and asked to complete a short survey (Fig. 2). Non-respondents were contacted a second time by email and finally by a paper copy through regular mail.

Fig. 2. Four-month follow-up survey.

UNIVERSITY OF TORONTO - FACULTY OF MEDICINE DIVISIONS OF EMERGENCY MEDICINE CME COMMITTEE

Faculty Development Workshop:
Clinical Teaching in the Emergency Department

Thank you for participating in the above workshop. In order that we may comply with accreditation regulations, we kindly ask that you complete the following post workshop survey. Certificates verifying your participation will be sent upon receipt of this survey.

  1. List THREE strategies that you have attempted in your practice to effect good teaching as a result of having attended this workshop.
     
  2. Please describe how successful the interventions have been and provide details about how they might be improved or barriers to them addressed.
     
  3. Have you tried to eliminate or improve upon any behaviours since attending the workshop? Please elaborate, including the degree of success you have had.
     
  4. How has your comfort level with clinical teaching in the emergency department changed since attending the workshop?
     
  5. Are there any elements of your clinical teaching encounters that remain unaddressed?
     
  6. Please comment on what you liked about the workshop and what you think should be changed.
     

The program was supported by the local Divisional Continuing Medical Education committee and was held at no cost to the participants. Breakfast and lunch were served. This study received institutional ethics review board approval.

Results

The course was fully subscribed within 24 hours of the initial advertisement. The 3 breakout session themes identified by the needs assessment were: "Common mistakes in ED teaching," "How to teach the difficult resident" and "How to teach efficiently." Table 1 summarizes the results of the same-day program evaluation. All 15 participants said they would attend future workshops on similar topics and all 15 said they would recommend the workshop to colleagues. Ten of the 15 faculty responded to the 4-month follow-up survey. All 10 had successfully implemented new strategies since the workshop. Eight of 10 said they were more comfortable teaching, and 2 felt that they were already comfortable before the workshop. The most commonly incorporated new strategies were setting specific learning objectives (7/10), giving focused feedback (5/10), actively seeking out learners (4/10) and focus on a single teaching point (3/10). Most common behaviour modifications included improving listening skills (4/10) and giving more and better feedback (3/10). Table 2 includes a list of all new strategies and improvements.

Table 1. Responses to the post-workshop questionnaire*
Evaluation domain Specific question content Rating†
Content items Relevance of the workshop to your practice 4.6
Tailored to individual needs 4.4
Time spent in interactive learning 4.1
Style of presentation 4.0
Usefulness of the content 4.5
Process items Pre-workshop notification and communication 4.4
Organization (clarity, agenda, logistics) 4.1
Knowledgeable faculty 4.3
Approachable faculty 4.4
Appropriateness of teaching aids (AV, overhead, etc.) 3.3
AV = audiovisual
*See Figure 1 for the questionnaire.
†Rating system: 1 = poor; 2 = needs improvement; 3 = as expected; 4 = better than expected; 5 = exemplary

Discussion

Our study demonstrates that EM faculty development initiatives based on a sound needs assessment and using material from relevant original research can lead to behaviour change. The 3 identified areas of interest, How to teach difficult learners, How to teach efficiently and Common mistakes in teaching, are in keeping with previous studies highlighting difficulties in ambulatory teaching.3,7,9,10,17-20

Teaching models from other ambulatory settings address common faculty needs, but are not tailored to EM teaching. The ED specifically challenges teachers by simultaneously introducing multiple learners at different levels from different programs.19,21 Teaching must occur in an unpredictable environment full of interruptions in which high-stakes decisions are made.10,17,21,22 All of these conspire to make personalized objective-setting difficult and may lead to a superficial teacher-learner relationship. For example, basing learning objectives on the planned daily roster, iterative priming for patient encounters, longitudinal progression in learner autonomy, and identifying learning objectives for a given patient in anticipation of a return visit are all strategies presented in teaching models that would need significant alteration for ED applicability.2,3,6,23 Our workshop was based on material from expert EM teachers to allow participants to address their major needs while considering the unique constraints of the ED.9 The credibility of the program was enhanced because the strategies provided were derived from the same environment in which the participants practise.

The behaviours reported at 4-month follow-up (Table 2) address the learning needs declared at the outset of the program. This is further indication of the success of the workshop. For example, improving feedback techniques, adapting learner autonomy and patience with slower learners are key in managing many difficult learners. Having a good knowledge of the learner as a person, understanding their learning needs, having pre-arranged material for teaching, and encouraging self-directed learning are all important for teaching efficiently. Finally, participants listed behaviours that they modified to avoid common mistakes, such as not listening to the learner.

Table 2. Responses to the 4-month follow-up survey* (n= 10)
Question Responses†
1. List 3 strategies you have attempted in your practice to effect good teaching as a result of having attended the workshop. Not to interrupt students.
Focus on 1 or 2 teaching points per case.
Seek students out.
Ask trainees what they are hoping to learn.
More hands-on demonstrations at bedside.
Encourage trainees to problem solve.
Trainees to use learning resources in the department and learn on their own.
You can place some responsibility for learning on the learner.
Having a teaching file ready when the department is slow.
Try to be more aware of learning stages, students, junior, senior, as well as individual levels.
Let fellows run the code/trauma.
Give feedback at the end of the shift.
3. Have you tried to eliminate or improve upon any behaviours since attending the workshop? Improved listening skills when reviewing cases, including eye contact.
Not talking during first minute of the case presentation.
Not saying "good shift" without giving feedback - improved how often and when I give feedback.
Have been more supportive of students having difficulty grasping a certain area.
Asked trainees something about themselves - the majority of the time.
Showed interesting findings to trainees when it's not their case.
Tried to be more patient, sending students off to get the answer on their own.
Realized it's okay to say the department is too busy right now - it's better to review cases after the shift.
Tried to relax a bit, but a work in progress.
*See Figure 2 for the follow-up survey.
†Some responses were listed by more than 1 respondent.

Our program used several techniques to encourage active learning. In one small group session, participants used the workshop reference manual to research potential solutions for teaching challenges identified in the pre-workshop assignment. An informal interactive discussion then followed to allow participants to think about how they might incorporate the ideas. This method allowed participants to demonstrate to themselves how best to use the manual as a reference to address their ongoing teaching challenges. Promoting discussion among colleagues allowed participants to learn how others would apply a newly learned strategy and emphasized the "grass-roots" origin of much of the manual. In a second small group, participants brainstormed and prioritized a list of teaching mistakes they had witnessed. The list was then compared with one generated in a national survey.9 This method encourages participants to identify teaching shortcomings without vulnerability and demonstrates how "real" the mistakes are in that even accomplished teachers fall victim to them from time to time. Learning what other participants identify as mistakes may validate one's own personal experiences or uncover previously unperceived mistake behaviours. Hearing a teaching error discussed among colleagues may have more personal impact than reading about it. Participants shared anecdotes to reinforce the points and, finally, brainstormed methods to avoid mistakes. In the final small group, participants role-played the approaches to difficult learners using trained EM residents as "learners." Feedback was provided by other participants, the residents, and the session leader. Interactivity is important in retention of new knowledge and skills,11,14,24 and learners rated the interactivity of the workshop highly.

The need to encourage excellence in teaching and provide resources for faculty development is more prevalent than ever. Today's medical learners have a greater sense of consumerism and look for accountability and quality in their education. The ED, a difficult teaching environment at the best of times, is becoming more difficult because of problems such as overcrowding.25 Finally, the Royal College of Physicians and Surgeons of Canada's CanMEDS framework specifies that residents must be trained communicators and scholars.26 Teaching both of these competencies requires that teachers be effective role models during their teaching. Feedback from the post-workshop survey confirmed a high perceived need for faculty development programs focused on ED teaching. All participants said they would recommend the course to colleagues and take a more advanced course. The average rating for "relevance" was 4.6/5. Many EM teachers feel unprepared to teach, a sentiment also found in many other medical specialties.12,15,27 Our results support this idea and demonstrate the desire for further faculty development in this area. We are encouraged by the fact that even self-selected teachers reported a positive impact of our workshop on their teaching behaviours. Our needs assessment was an important step in informing content development, and we encourage others to include this step to help tailor similar projects to local needs.

Limitations

The primary limitation is that we relied on self-reporting to assess behaviour change. Although respondents reported multiple beneficial effects of the workshop, we cannot determine whether these are actual or perceived changes. Subsequent focus groups with students suggest many of the reported changes are real and beneficial, but we did not formally measure the effects of new teaching behaviours on relevant learning outcomes and we cannot attribute perceived improvements in learning to our workshop.10 Our results are, therefore, in keeping with level 1 in Kirkpatrick's 4-level evaluation model.28 Finally, the number of faculty in this study is small and effects may not be generalizable to a broader audience. Although our faculty came from university-affiliated teaching hospitals and had academic appointments within a division of EM, they were volunteers with no financial incentive or protected time to attend. Our impression is that most Canadian emergency physicians engaged in teaching would fit this description. Further assessment of the generalizability of this initiative will be forthcoming, as it has formed the basis for a larger, highly flexible needs-based national faculty development program for ED teaching (ED STAT!: Strategies for Teaching Any Time; www.caep.ca).

Conclusions

Emergency medicine faculty report learning new strategies for teaching from a faculty development workshop based on a formal needs assessment and original research on ED teaching. The workshop met perceived needs. The most commonly implemented new strategies were tailoring teaching to the learner, providing better feedback, and actively involving learners.

References

  1. Gallagher EJ, Martin M. Faculty development handbook. Lansing (MI): Society for Academic Emegency Medicine; 2000.
  2. Heidenreich C, Lye P, Simpson D, Lourich M. The search for effective and efficient ambulatory teaching methods through the literature. Pediatrics 2000;105:231-7.
  3. Irby DM. Teaching and learning in ambulatory care settings: a thematic review of the literature. Acad Med 1995;70:898-931.
  4. Irby DM, Ramsey P, Gilbert D, Shaad M. Characteristics of effective clinical teachers of ambulatory care medicine. Acad Med 1991;66:54-5.
  5. Ferenchick G, Simpson D, Blackman J, De Rosa D, Dunnigton G. Strategies for efficient and effective teaching in the ambulatory care setting. Acad Med 1997;72:277-80.
  6. Neher JO, Gordon K, Myer B, Stevens M. A five-step "microskills" model of clinical teaching. J Am Board Fam Pract 1992;5:419-24.
  7. Skeff KM. Enhancing teaching effectiveness and vitality in the ambulatory setting. J Gen Intern Med 1988;3(2 suppl):S26-33.
  8. Stritter FT, Baker RM. Resident preferences for the clinical teaching of ambulatory care. J Med Educ 1982;57:33-41.
  9. Bandiera G, Lee S, Tiberius R. Creating effective learning in today's emergency departments: how accomplished teachers get it done. Acad Emerg Med 2005;45(3):253-61.
  10. Bandiera G, Thurgur L, Lee S, Tiberius R. What Emergency medicine learners wish their teachers knew. Clin Invest Med 2004;27:201.
  11. Steinert Y, Nasmith L, Daigle N. A faculty development workshop on "Developing Successful Workshops." Acad Med 2000;75:5545.
  12. Skeff KM, Stratos G, Mygdal W, Dewitt T, Manfred L, Quirk M, et. al. Faculty development. A resource for clinical teachers. J Gen Intern Med 1997;12 (suppl 2):S56-63.
  13. Coates WC, Hobgood CD, Birnbaum A. Faculty development: academic opportunities for emergency medicine faculty on education career tracks. Acad Emerg Med 2003;10:1113-7.
  14. Hamilton GC, Brown JE. Faculty development: What is faculty development? Acad Emerg Med 2003;10:1334-6.
  15. Wilkerson L. Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med 1998;73:387-96.
  16. Ramalanjaona G. Faculty development: how to evaluate the effectiveness of a faculty development program in emergency medicine. Acad Emerg Med 2003;10:891-2.
  17. Chisholm CD, Collison E, Nelson D, Cornell W. Emergency department workplace interruptions: Are emergency physicians "interrupt-driven" and "multitasking"? Acad Emerg Med 2000;7:1239-43.
  18. Irby DM. What clinical teachers in medicine need to know. Acad Med 1994;69:333-42.
  19. Penciner R. Clinical teaching in a busy emergency department: strategies for success. Can J Emerg Med 2002;4(4):286-8.
  20. Skeff KM, Bowen JL, Irby DM. Protecting time for teaching in the ambulatory care setting. Acad Med 1997;72:694-7; discussion 693.
  21. Carter AJE, McCauley WA. Off-service residents in the emergency department: the need for learner-centeredness. Can J Emerg Med 2003;5(6):400-5.
  22. Taylor AS. Emergency medicine educational objectives for the undifferentiated physician. J Emerg Med 1994;12:255-62.
  23. Furney SL, Orsini A, Orsetti K, Stern D, Gruppen L, Irby D, et al. Teaching the one-minute preceptor. A randomized controlled trial. J Gen Intern Med 2001;16(9):620-4.
  24. Laidley TL, Braddock IC. Role of adult learning theory in evaluating and designing strategies for teaching residents in ambulatory settings. Adv Health Sci Educ Theory Pract 2000;5:43-54.
  25. Atzema C, Bandiera G, Schull MJ. Emergency department crowding: the effect on resident education. Ann Emerg Med 2005;45(3):276-81.
  26. Societal Needs Working Group. Skills for the new millennium. Ann RCPSC 1996;29:206-16.
  27. Skeff K. Clinical teaching improvement: past and future for faculty development. Fam Med 1997;29:252-7.
  28. Kirkpatrick D. Evaluating training programs: the four levels. San Francisco: Berrett-Koehler; 1994.