There are positive and negative aspects associated with the
lists of general competencies and specialty-specific milestones set forth by
the Accreditation Council for Graduate Medical Education (ACGME).
On the plus side, these well intended, high-minded and
honorable lists of competencies ratify and draw attention to important areas of
professional performance. As educators, we should be accountable to the public
for assuring that trainees bearing our imprimaturs have achieved necessary
levels of competence prior to independent practice.
On the minus side, a “competency-industrial complex”
requiring formal assessment has emerged, generating large numbers of
bureaucratic documentation requirements, chock-full of unintended consequences
and unfunded mandates with regard to effort, time and staff resources.
The trick for educators and program staff is to
(competently) conduct the necessary appraisals and fulfill documentation
requirements without killing the spirit of educating trainees in the process.
Competency assessment should not turn into mind-numbing, cynical exercises
devoted to filling out valueless forms. Nor should formal evaluation stand in
the way of (or stand in for) actually guiding progressive competence-building
The following tips are provided with two goals in mind: 1)
Helping trainees grow their competence AND 2) Satisfactorily documenting their
5 Tips for Helping Trainees Increase Competence
- Appreciate that the majority of trainees accepted into our
programs ultimately meet expected levels of competence. When trainees know
what’s expected of them (i.e., are given the lists of expectations at the
outset), most can acquire these abilities and, when necessary for Board
certification, document that they’ve competently performed them. As educators
we should approach trainees with the attitude and expectation that they will
meet these competencies. Moreover, we should know what these expectations are
in order to competently guide them toward achievement.
- Try not to generalize or set expectations about knowledge or
skill-based competence from single observations, especially early in the
trainee’s experiences. Appreciate that with repeated practice, demonstration,
and feedback based on close faculty observation and encouragement, most
trainees who seem initially to be inept ultimately turn out “ept.”
- Use your specialty’s competency milestone statements as the
basis for offering trainees constructive comments for improvement as well as
positive feedback. Do so immediately after you observe a trainee’s performance
or as soon thereafter as feasible.
- Understand that a single observation of frank unprofessional
behavior is worrisome. Acts of frank unprofessionalism may reflect enduring
personality issues, insecurity, or counterproductive responses to clinical
pressures or workplaces stresses. Any act should be discussed with the trainee
immediately and should serve as the basis for close scrutiny in the future.
- Understand that medical educators must act responsibility
concerning the small number of trainees who are unable to master the requisite
competencies and who act unprofessionally. The ultimate public trust mandates
that medical educators protect patients and their families from incompetent
practitioners. There are resources available to support remediation of
struggling learners through the Office of Lifelong Learning (contact Jeannette Guerrasio). The earlier you identify the issue, the better we are able to help
improve the long-term outcome.
Tips for Formally Evaluating Competencies
- Unless these are “high stakes” ratings, don’t waste much
time developing your own competency or milestones assessment forms. (Unless
you’re planning to devote yourself to competence assessment as a major, time-consuming
scholarly project!). Instead, use forms that have been developed and
disseminated by professional educational associations or other educators in
your specialty. Of note, if you have the opportunity to engage in
cross-institution projects to develop and test competence assessment tools, by
all means, participate if the project will not take up an inordinate amount of
time and you are interested in the professional development and
community-building aspects. You may benefit professionally as a result, while
enjoying the company of educators from other institutions participating in the
- Remember that detailed narrative descriptions are generally
more valuable than numerical rating systems. However, many busy faculty fail to
take sufficient time, or are insufficiently articulate, to offer nuanced
accounts. Of note: faculty provided with detailed, descriptive “drop down”
menus of complete sentences/paragraphs that capture important performance
characteristics may help raters save time and ultimately provide program
directors with higher quality information on which to act. Regardless, your
comments and specific examples as a faculty member who has directly observed
the behavior of the resident are INVALUABLE.
- Use the competency-based forms and milestone instruments as
prompts for helping to organize your thinking about residents’ performances. In
day to day activities, think about how each trainee’s performance aligns with
the various expected competencies and milestones.
- Understand that, except for specific procedural skills and
at the extremes, most rating schemes of general
domains (e.g., “medical knowledge”, “systems-based care”) on which
faculty are asked to evaluate trainees have limited reliability and validity
because they are broadly stated and poorly operationally-defined. Realize that
most assessments are inexact. Except in cases of frank incompetence or
exceptionally brilliant performance, you’re unlikely to encounter strong
inter-rater reliability (not that such reliability is guaranteed even at the
extremes). Whereas gross assessments of “safe”, “unsafe,” “smart” and “dumb”
may be replicated, inter-faculty agreement on finer-grained distinctions within
each area of competency may be much harder to achieve.
- Appreciate that competency-based evaluation forms offer
trainees important opportunities for self-reflection and self-assessment.
Trainees should be asked to self-evaluate using these forms and to discuss
their self-evaluations with faculty. Many trainees humbly undervalue
themselves, often rating their competence as lower than perceived by faculty.
On the other hand, finding trainees with an inflated sense of competence is
potentially worrisome and may warrant closer future scrutiny.
Before Completing a Rating Form
- Know exactly who will see the form and how it will likely be
used—now and in the future. What are the intended consequences of their use,
and what unintended consequences might you foresee – on the trainee’s
self-esteem, reputation, future recommendations and future prospects? How might
any of your comments come back to you in the future? Before you err on the side
of harsh evaluations, for example, do you feel sufficiently justified in the
strength of the evidence on which you base your opinions (e.g., repeated
observations or an egregious sentinel event) so as to rate someone in a strong,
negative manner? Have you sufficiently documented the specifics involved so
that the person receiving the evaluation can appropriately use the information
to act on the information provided?
- Know your own rating biases. Are you a generally hard rater
or a generally soft rater? Are you
taking these biases into account?
- Think about the rating biases of your peers and try not to
succumb to “groupthink.” Don’t allow yourself to be subconsciously biased or
swayed because you’ve heard other faculty badmouthing or excessively praising a
trainee. For your own assessments, do your best to discount others’ comments
and base your assessments entirely on your own observations and interactions.
Be honest when you’ve had insufficient experience with a trainee to offer a
We hope you find these tips useful as you work on the
assessments required of you. Note that the Academy of Medical Educators is
happy to come to your department, division, section or program meetings to
provide faculty development training specific to your assessment framework. If
you are interested, please contact Eva Aagaard.