STUDENT  EVALUATION  INSTRUMENT  COMMITTEE
OF  THE  ACADEMIC  COUNCIL

Original Committee Membership (98-99): Larry Basirico, John Burbridge, Jim Drummond, Dave Nawrocki (chair), Maurice Levesque, Deborah Thurlow (former member)

Current Committee Membership:  Larry Basirico (chair), John Burbridge, Jim Drummond, Maurice Levesque
 

  I.     Problems with Current Instrument
         The Ad Hoc Committee on the Student Evaluation Instrument was organized
         in the spring of 1998 to review the current student evaluation of faculty
         instrument and to propose any needed changes. The committee identified
         the following problems with the current instrument:

        1.   Current instrument is too long in terms of administration and providing feedback that
              is useful for faculty (self-assessment), chairs and deans.

        2.   Current instrument is difficult to use comprehensively for both evaluative and
              formative purposes.

        3.   Current instrument has redundant, ambiguous, or irrelevant questions, which makes
              it difficult to measure the quality of teaching.

         4.  Current instrument lacks questions that measure essential teaching qualities, which can
              be applied across all of the disciplines.

        5.   On the current instrument, it is not clear what the single indicator question ("Overall,
              I rate this a good instructor") measures and it may not necessarily be a measure of
              quality teaching.

        6.   Currently, there is a lack of uniformity in the control and administration of the
                instrument.
 

 II.    Principles for developing new instrument

         Keeping in mind the above problems, the committee developed a new instrument
         based upon the following principles:

        1.   Instrument should be based upon elements of quality teaching that traverse all
              disciplines. Much of this was formed by the standards of assessing quality teaching
              presented by Linda Nilson.

        2.   Questions should be clear and unambiguous.

        3.   Results of the instrument should be more easily accessible both for evaluative and
              formative purposes.

        4.   Any single indicators of quality of teaching should not be based upon a single question
              but, rather, be based upon a summary of all elements of quality teaching addressed by
              the instrument.

        5.   Evaluation of the course should not be included with an evaluation of the instructor.
              The committee recognizes that evaluation of courses may be of importance for
              programmatic reasons, but feels that since the student evaluation instrument plays a part
              in faculty member's evaluation with regard to raises, promotion, tenure, and
              professionalization, faculty members should neither suffer nor benefit from any
              evaluations stemming from the course itself.

        6.   The instrument should address items that students are in a position to evaluate. For
              example, students are not in a position to evaluate whether or not the readings or the
              type of assignmetns are appropriate to the level of the course. This suggestion also
              comes from Linda Nilson.

        7.   Administration of the instrument should include specific instructions about when the
              instrument should be administered, instructions as to the class conditions (e.g., not
              to be administered when returning papers or exams), and a specific statement to be
              read by the administering person. The committee considered the possibility of
              requesting that administering the instrument be by trained personnel (e.g., secretaries,
              advising staff, honors students,e tc.) and would like to call attention to the benefits.
              However, the committee also recognizes the logistical problems associated with
              this and, therefore, does not offer it as a specific part of its overall recommendation
              at this time.

        8.   The results of the instrument should include a mean scale for each item as well as
              the standard deviation, an indicator of the range of responses given.
 

III.    Potential Shortcomings for proposed instrument

         The committee recognizes that there are still shortcomings with the proposed
         instrument, but also that the benefits of the proposed instrument outweigh
         these shortcomings. The committee identified the following shortcomings:

         1.   Each item is given equal weight. The committee recognizes that different courses
                and different disciplines may give different weight to the items depending upon the
                situation. However, our goal was to develop an instrument that could be used
                college-wide and to identify elements of quality teaching that all professors should
                strive toward mastering.

        2.   The proposed instrument may not address teaching qualities that may be specific to
              some disciplines or some courses. Once again, our intent was to develop an
              instrument broad enough to cut across all disciplines without introducing items that
              might be irrelevant to other disciplines or courses. To mitigate against failing to
              address qualities that may be deemed important within a specific discipline, there will
              be room on the proposed instrument for additional items. However, the committee
              proposes these items be excluded from the summary indicator and be used primarily
              for formative purposes.

        3.   The proposed instrument does not include an evaluation of the course. As noted above,
              this omission was intentional. The committee recommends that if course evaluation is to
              take place, it should be a separate process from the evaluation of faculty.

        4.   The committee recognizes the omission of any questions that measures the extent to
              which students feel they have achieved the learning objectives. However, this omission
              was intentional since the committee feels that student performance in a class is not
              necessarily under the control of the faculty.
 

 IV.   Pilot Study

 The proposed instrument was offered as a pilot to 25 sections in Spring, 1999 (N=634
students). Professors were asked to report, informally, about their experiences with the
proposed instrument and for any feedback. Responses were highly favorable.

The results of the pilot survey were calculated for each course taught by each professor and
for all courses (25) combined. The results (attached) are of the combined results so as to
respect the anonymity promised to the professors.

Some general comments about the statistical results:
 



Sample of evaluation form used Spring 1999

Results of  the new evaluation forms, Spring 1999


Return to the 1999-2000 Council Business Page.

Return to the Academic Council Home Page.