Multi-perspective usability evaluation with multi-criteria decision analysis for optimal selection of open-source software

Increasing demand for open-source software (OSS) has raised the neat of efficient selection in terms of quality; usability is an essential quality factor that significantly affects system acceptability and sustainability. Most comprehensive and complex software packages are partitioned across multiple portals and involve many users — each with their own role in the software package. Those users have different perspectives on the software package defined by their knowledge, responsibilities and commitments. Thus, a multi-perspective approach has been used in usability evaluation to overcome the challenge of inconsistency between users’ perspectives, which would lead to an ill-advised decision on the selection of a suitable OSS. This research aims to assist public and private organisations in evaluating and selecting the most-suitable OSS. The selection of the best OSS based on usability evaluation criteria is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, and (c) data variation. Thus, it is considered a sophisticated multi-criteria decision making (MCDM) problem. A generally accepted multi-perspective usability evaluation method for the selection of OSS is unavailable in the existing literature. Hence, this research proposes a methodology for multi-perspective usability evaluation with multi-criteria decision analysis for optimal selection of open-source software. Integration of the best-worst method (BWM) and VIKOR MCDM techniques have been used for weighting and ranking OSS alternatives. BWM is utilised for weighting of evaluation criteria, whereas VIKOR is applied to rank OSS-LMS alternatives. Individual and group decision-making contexts and the internal and external groups’ aggregation were used to demonstrate the proposed methodology's efficiency. A well-organised algorithmic procedure is presented in detail, and a case study was examined to illustrate the validity and feasibility of the proposed methodology. The results demonstrated that the proposed methodology works effectively to solve the OSS selection problem. Furthermore, the ranks of OSS software packages obtained from the VIKOR internal and external group decision making are similar; the best OSS-LMS based on the two ways was ‘Moodle’. Among the groups' scores in the objective validation, significant differences were identified, indicating that the ranking results of internal and external VIKOR group decision making were valid, thus validating the proposed methodology.

Text FSKTM 2021 5 - IR.1.pdf Download (2MB)

Actions (login required)

View Item