UPM Institutional Repository

Setting standards for judging the rapid film reporting section in postgraduate radiology assessment: a feasibility study


Abdul Rahim, A. F. and Roslan, N. S. and Shuaib, I. L. and Abdullah, M. S. and Sayuti, K. A. and Abu Hassan, H. (2021) Setting standards for judging the rapid film reporting section in postgraduate radiology assessment: a feasibility study. Hong Kong Journal of Radiology, 24 (1). 23 - 30. ISSN 2223-6619; ESSN: 2307-4620


Objective: We investigated the feasibility of applying a standard-setting procedure for the rapid film reporting examination of the Malaysian National Conjoint Board of Radiology. Methods: We selected the modified Angoff standard-setting process. Judges were nominated and trained, performance categories were discussed, and judges’ ratings on films were collected after an iterative procedure. The process was then evaluated for evidence of validity. Results: A cut-off score of 92% resulted, compared with the 80% usually used. Judges were satisfied with the training and understood the procedure and their roles. In all, 27.3% felt that time given for the task was not sufficient. A total of 54.5% of judges thought the final passing cut-off score of 92% was too high, and 27.3% were not confident regarding its appropriateness. The inter-rater reliability was 0.928. External comparison with a ‘gold standard’ of supervisor ratings revealed a sensitivity of 0.25 and specificity of 1.00 compared with the traditional cut-off score having a sensitivity of 0.92 and specificity of 0.33. In this kind of situation, high specificity is considered to be more important than high sensitivity. Conclusion: Standard setting for the rapid film reporting examination using the modified Angoff method was feasible with robust procedural, internal, and external validity evidence. Areas for improvement were identified to address the perceived high cut-off score obtained and improve the overall process.

Download File

Full text not available from this repository.
Official URL or Download Paper: https://www.hkjr.org/article/v24n1/23

Additional Metadata

Item Type: Article
Divisions: Faculty of Medicine and Health Science
DOI Number: https://doi.org/10.12809/hkjr2117199
Publisher: Hong Kong Academy of Medicine Press
Keywords: Education; Medical; Preceptorship; Psychometrics; Radiology; Teacher training
Depositing User: Ms. Che Wa Zakaria
Date Deposited: 12 Apr 2023 04:15
Last Modified: 12 Apr 2023 04:15
Altmetrics: http://www.altmetric.com/details.php?domain=psasir.upm.edu.my&doi=10.12809/hkjr2117199
URI: http://psasir.upm.edu.my/id/eprint/95054
Statistic Details: View Download Statistic

Actions (login required)

View Item View Item