The evaluation system is the primary mechanism for collecting and using information to describe program effectiveness and identify areas for improvement in the Medical School Curriculum. The evaluation system consists of the:
- Curriculum Steering Committee (CSC) which oversees the content and methods for data collection, sets policies for compliance, and reviews evaluation results to determine areas of improvement.
- Evaluation Office (EO) which develops a comprehensive evaluation plan, proposes policies, designs evaluation instruments in conjunction with program faculty, administers evaluation instruments, and analyzes and reports results.
- Liaison Committee on Medical Education (LCME) which accredits medical school programs in the United States, sets standards for schools, including the need to evalute the effectiveness of the medical school curriculum.
- UC SOM faculty who teach in the medical school and Department Chairs who support faculty on the basis of their teaching effectiveness.
- UC SOM students who provide evaluation data and contribute to evaluation reports.
We gauge the overall effectiveness of medical school curriculum by evaluating its many different parts. This system generates evaluations of course blocks, small group sessions, clinical experiences such as rotations and clerkships, as well as, faculty instruction and student performance.
We collect a wide variety of information.
- Students' responses to questions about their attitudes, perceptions, and learning experiences are collected. Students provide information either online or paper questionnaires which usually are given soon after the end of a course or block. These data are supplemented by focus groups. Focus groups provide interactive, in-depth evaluations of individual courses and students' experience acoss courses. Focus groups are conducted at the end of each Block in Phases I and II and at the end of Phases II and Phase III. In addition to student reported experiences, faculty evaluations are supplemented by Block Directors' observations of lecturers. Students also provide an overall evaluation of the curriculum when they complete the Graduation Questionnaire administered by the AAMC at the end of Phase IV. These data permit comparisons to the national data.
- Indicators of student performance are examined as well. These data include student performance on individual course exams and faculty assessments of student in the clinical years. Student performance on the Clinical Practice Examination administer by CAPE provides rich information about students' clinical skills at the end of the third year. National data on board exams-USMLE Step 1 and Step 2- provide comparative performance data.
- Graduates' views on the program effectiveness of the Medical School Curriculum are collected 1, 5 and 10 years after graduation. Resident program directors for recent graduates are also surveyed every other year to assess the competency of our graduates.
Using this information, evaluation reports are produced for courses in the Essential Core Blocks (Phase I and Phase II), rotations in the Clinical Blocks (Phase III), and Sub-Internships and electives (Phase IV). All of this information is then used for the School of Medicine’s accreditation review conducted by the Licensing Committee for Medical Education (LCME) every 8 years.
Students' perspectives on the effectiveness of Blocks and individual faculty provide essential information for block improvement. Block evaluations are used by Block Directors and the Curriculum Steering Committee (CSC) to modify block content, organization, or instruction. Faculty evaluations are used to determine topics for faculty development, to provide assistance to individual faculty, and to inform promotion decisions.
During the first two phases, all students evaluate their courses, lecturers, and small group facilitators in the Essential Core Blocks, as well as the Foundations of Doctoring Course, and Problem Based Learning activities(PBL). To ensure coverage but reduce the burden on any individual student, students are randomly assigned to evaluate three Blocks and the lecturers in the Block, during Phase I and Phase II. During Phase III, students evaluate each Clinical rotation and site(s) within the Clinical Blocks, the attendings they worked with, and all Intersession Clinical Courses (ICC). In Phase IV students evaluate Sub-Internships and attendings, and all electives and ICCs.
In general, questionnaires are sent out immediately following the completion of a course or Block. Because evaluations are required, students must complete questionnaires within the given timeframe (for more detail see “What happens if a student fails to complete an evaluation?”). Focus groups in Phase I and Phase II are usually held near the end of a given course, and are scheduled to avoid conflicts with planned course and session meetings.
We know that some students worry about giving constructive feedback regarding a course or instructor. In order to ensure complete and accurate information, we take various steps to ensure student anonymity when completing required evaluations. In Phase I and II, evaluations are administered on-line using the CoursEval system at https://evals.ucdenver.edu. CoursEval tracks students’ names in one database and their responses in another. The separation of these databases helps to keep responses anonymous and secure. In Phase III and IV, students provide evaluations of course and faculty online through the New Innovations system. As a matter of policy, no evaluation data are reported until at least 3 students have evaluated the course or faculty member. In the Evaluation Office, we take student anonymity seriously, and strive to ensure students can feel confident in the system and thus provide complete and honest feeback about courses and teaching.
Block Directors voted to make evaluation a requirement because feedback on how well a Block works and the quality of teaching is essential to improving the curriculum. Student feedback can lead to shifts in course content or organization, development of new policies, or additional support for instruction. In addition, because evaluation is required, the burden of evaluation is reduced for all students since a smaller and randomly selected group of students can provide accurate, representative data about a Block. (For more, see, "Why do students evaluate Blocks and faculty members?")
An important learning outcome of medical education is "professionalism," and practicing the standards set by the field. A first step into the medical profession includes the responsibility to provide feedback and help improve medical education. Because of the importance of this responsibility, a set of steps have been put in place to encourage students to complete required evaluations.
Phases I and II(during Essential Core Blocks): Evaluations are required during the pre-clinical years. Students in the have 10 business days to complete their evaluations. Students who have not completed their evaluations(s) after 5 days receive email reminders; a final reminder is sent on the 9th working day. A student not completing a course evaluation in 10 working days will be reported to the Assistant Dean, Essentials Core. The Assistant Dean will contact the student, explain the importance of student feedback, the purpose of the course evaluation policy, and the consequences for failing to meet his/her responsibilities. If the student fails to complete a second evaluation, s/he will meet with the Assistant Dean to discuss the policy and a Professionalism Feedback Form will be filed. If the student fails to complete a third evaluation, s/he will meet again with the Assistant Dean to discuss the policy and a second Professionalism Feedback Form will be filed. As per the Professionalism Committee Process, when a repeated instance of unprofessional behavior is recognized, the student's behavior will be reviewed by the Professionalism Committee.
Phases III and IV: Evaluations are required during the clinical years. All students must evaluate each clinical rotation and attendings and residents they worked with in the Clinical Blocks and Sub-Internships. Students are required to evaluate all Phase IV electives. Evaluations are administered in the New Innovations sustem. Students are given 10 working days to complete evaluations. Failure to complete an evaluation results in the assignment of an IP (Incomplete Pass) for the course. Students cannot progress to the next rotation or elective until the evaluation is completed and the IP grade removed.
Evaluation policies are developed by the Evaluation Office in response to the recommendations of Block Directors as made in the Essentials Core Block Directors’ Committees (ECBD) or Clinical Block Directors’ Committees (CBD). Policies are then forwarded to the Curriculum Steering Committee (CSC) for review and approval.
The online CoursEval system produces evaluation reports for each Block and faculty teaching within a Block. Block Directors and faculty receive email notices when evaluation summaries are available and can access reports directly. Evaluation Office also sends Block evaluation data to the education Deans (i.e., Sr. Associate Dean for Education, Associate and Assistant Deans for Curriculum), the Chairs of the CSC, and Block Directors. Block evaluation information is summarized by the Director of Evaluation and integrated into a Block Summary Report by the Block Directors with the student representatives for the block and can be found on the web page with the committee minutes. [LINK TO BE ADDED]
The Block Summary Reports are presented to Block Directors in the Essential Core Block Directors' Committee (ECBD) or in the Clinical Block Directors' Committee(CBD) to communicate what works and to improve coordination across blocks. The reports are used by the Curriculum Steering Committee (CSC) to monitor the curriculum, modify policies, recommend changes, and provide faculty development.
The Evaluation Office prepares annual reports for each phase of the curriculum as well as comparative data from alumni, interns, and graduating seniors. Comparative data are reported routinely to the CSC to monitor the curriculum.
Finally, these data are used for accreditation purposes. They form the basis of database for a self-study prepared by the School of Medicine. The database and self-study report are provided to an external review committee from Licensing Committee for Medical Education (LCME) to help them determine whether the School meets curriculum and evaluation standards for accreditation. The next LCME site visitation will be held in May, 2011
Once approved by the CSC, reports are available to all members of the UC SOM community (faculty and students) to communicate the status of the curriculum. [LINK TO BE ADDED]
|
|