Consequential validity of student ratings of instruction for administrative purposes and instructional improvement

Student ratings of instruction (SRI) are popular evaluative procedures which are widely practiced in universities around the world including most Malaysian universities. With many universities using student ratings around the world, there are still uncovered issues regarding student ratings that research has not yet explored sufficiently. For instance, student ratings of instruction have been the subject of several validity studies with much of the research focusing on traditional evidence of validity. Only a fewer studies have examined consequential validity of SRI. Validity in its modern concept focuses on the use of instrument for a particular purpose as well as the consequences of instrument use. The evidence on consequences can include evaluation of the effects. Two intended effects of using student ratings are making instructional improvement by faculty members and making administrative decisions by administrators. This study examined consequential validity of student ratings by investigating evidence on the consequences as well as evidence on the utility of student ratings based on the Messick’s validity theory. Additionally, this study uncovered the administrators’ and faculty members’ reactions toward student ratings and obtained their recommendations for the effective use of student ratings. In this study, a descriptive survey design was employed. Two self-developed instruments were used to collect data from a stratified proportionate random sample of 141 administrator and 1089 faculty member populations. The responses were subjected to descriptive statistics, principal component analysis, Pearson’s Product Moment correlation analysis, one way ANOVA, and independent t-test. The study explored the perceived actual and the perceived potential consequences of student ratings on one dimension of administrative practices and two dimensions of instructional practices. Findings showed that the ratings information which focused on faculty members’ efforts in student learning enhancement was considered by both administrators and faculty members as the most useful type of ratings information. These findings addressed the consequential validity of student ratings at UPM, supported/improved validity of the UPM’s student ratings instrument,addressed a gap in the validity of student ratings in its general conception in the literature and in its specific conception at UPM, and provided the university with a reliable evidence for justifying the use of student ratings. This study revealed that administrators and faculty members had positive views toward student ratings. The findings also listed their recommendations regarding issues, such as, supplementing the ratings scale with other measures of teaching quality; providing opportunities for less effective faculty members to receive consultation services; reporting the result of student ratings in a multi-dimensional feedback form; providing an array of items so that different faculties/departments could select items suitable for their needs; and providing written explanations to assist faculty members in interpreting the results. It is expected that the findings of this research will bring valuable insight and understanding to help the university authorities at UPM specifically, and other universities generally, as well as all faculty members to achieve more success in using the student ratings, and consequently, in enhancing the quality of instruction.

Preview PDF FPP 2012 22R.pdf Download (978kB) | Preview

Actions (login required)

View Item