Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities

Rating scales are useful tools that can mitigate the validity issues of objective writing tests. In addition, they can reduce the reliability problem of essay tests that are scored impressionistically. The existing instruments are not appropriate for use in Malaysian universities. They are either generic or their grading system is not suitable for this context. The only genre-specific scale developed in Malaysia (Wong, 1989) is suitable for the narrative mode. The lack of adequate scales leads the lecturers to score their learners‟ written pieces impressionistically (Mukundan & Ahour, 2009). This developmental study included design, operationalization, trial and validation of an analytic scale of argumentative writing. ESL writing lecturers‟ views on the importance, wording and inclusiveness of the criteria were investigated (through qualitative and quantitative methods) with the help of a checklist. Parallel with these studies, a number of argumentative samples (n =20) were also analyzed. Thes samples had been selected from a batch of 167 argumentative essays written by students from six different faculties of Economy and Management, Health and Medicine, Design, Communication, Agriculture and Ecology in Universiti Putra Malaysia (UPM). Additionally, a focus group study helped the researcher further refine the checklist based on its respondents‟ views. They included four female senior lecturers from a Malaysian public university. As a result, a prototype scale was developed. Focus group participants reconvened to further refine the prototype, the clarity of its rubrics and their weights, the anchor papers and extended scoring guide. The result was the Analytic Scale of Argumentative Writing (ASAW). It followed an equal-weight scheme with five subscales of „content‟, „organization‟, „language conventions‟, „vocabulary‟ and „overall effectiveness‟ and could diagnose five levels of performance. The scale was tested for its inter/intra-rater and internal reliability, which resulted in moderate/high coefficients. Concurrent validity tests showed high and significant relationships between the students‟ MUET bands and the scores assigned to their written samples using ASAW. In addition, the samples were scored using four other well-established writing scales. Correlation tests indicated moderate and significant correlations between these scores and those assigned using ASAW. As for consequential validity, the raters‟ responses to a questionnaire on usefulness of ASAW indicated their moderate-very high levels of satisfaction with it. The scale can be used as a tool to benefit three main contexts in language instruction “(a) research, that is, theory building; (b) policy development; and (c) professional practice, that is, classroom or curriculum development” (Kiely, 2009:101).

PDF FPP_2010_13_F.pdf Download (235kB)

Actions (login required)

View Item