UPM Institutional Repository

A comparative study in classification techniques for unsupervised record linkage model


Ektefa, Mohammadreza and Sidi, Fatimah and Ibrahim, Hamidah and A. Jabar, Marzanah and Memar, Sara (2011) A comparative study in classification techniques for unsupervised record linkage model. Journal of Computer Science, 7 (3). pp. 341-347. ISSN 1549-3636; ESSN: 1552-6607


Problem statement: Record linkage is a technique which is used to detect and match duplicate records which are generated in data integration process. A variety of record linkage algorithms with different steps have been developed in order to detect such duplicate records. To find out whether two records are duplicate or not, supervised and unsupervised classification techniques are utilized in different studies. In order to utilize the supervised classification algorithms without consuming a lot of time for labeling data manually, a two step method which selects the training data automatically has been proposed in previous studies. However, the effectiveness of different classification techniques is the issue which should be taken into accounts in record linkage systems in order to classify records more accurately. Approach: To determine and compare the effectiveness of different supervised classification techniques in an unsupervised manner, some of the prominent classification methods are applied in duplicate records detection. Duplicate detection and classification of records in two real world datasets, namely Cora and Restaurant is experimented by Support Vector Machines, Naïve Bayes, Decision Tree and Bayesian Networks which are regarded as some prominent classification techniques. Results: As experimental results show, while Support Vector Machines outperforms with F-measure of 96.27% in Restaurant dataset, for Cora dataset, the effectiveness of Naïve Bayes is the best and it leads to an improvement with F-measure of 89.7%. Conclusion/Recommendation: The result of detecting duplicate records with different classification techniques tends to fluctuate depending on the dataset which is used. Moreover, Support Vector Machines and Naïve Bayes outperform other methods in our experiments.

Download File

[img] PDF
Restricted to Repository staff only

Download (134kB)

Additional Metadata

Item Type: Article
Divisions: Faculty of Computer Science and Information Technology
DOI Number: https://doi.org/10.3844/jcssp.2011.341.347
Publisher: Science Publications
Keywords: Bayesian network; Classification techniques; Data integration; Duplicate detection; Heterogeneous data; ID3 algorithm; Longest Common Subsequence (LCS); Optical character recognition (OCR); Record linkage; Support vector machines
Depositing User: Nabilah Mustapa
Date Deposited: 08 Jun 2016 08:32
Last Modified: 08 Jun 2016 08:32
Altmetrics: http://www.altmetric.com/details.php?domain=psasir.upm.edu.my&doi=10.3844/jcssp.2011.341.347
URI: http://psasir.upm.edu.my/id/eprint/22451
Statistic Details: View Download Statistic

Actions (login required)

View Item View Item