Hand gesture classification using convex hull skeleton joints moment k-nn technique for dimensionality reduction and incremental learning in the central nervous system’s hologram

Recent breakthroughs with numerous visual experiences using mobile devices encourage the research of human-computer interaction (HCI) involving hand gesture recognition for Holograms, Virtual Reality, and Augmented Reality. The rise of these technologies allows educators in medical segments to apply new pedagogy by interacting with virtual content in a coherent learning environment. In this thesis, the Central Nervous System (CNS) interaction is implemented using the Skeleton Joints Moment (SJM) approach for data reduction and convex hull k Nearest Neighbour (Convex Hull k-NN) for hand gesture classification. Principal Component Analysis (PCA) is commonly used for dimensional reduction as a data preprocessing for machine learning like k-NN, Support Vector Machine (SVM), and Artificial Neural Network. However, PCA implementation requires recalculation for a new batch of data. Therefore, this thesis presented the SJM CH k-NN with the Density Mapping to classify hand gestures in CNS application. Evaluation results show that this method supports incremental learning with optimized classification complexity than PCA k-NN, SVM, and ANN. This thesis introduces SJM CH k-NN with Density Mapping that addresses three hologram interaction issues using low-end mobile devices. The issues are data dimensionality, the complexity of hand gesture classification for incremental learning, and the uncertainty of hand gesture classification within a class intersection. First, this thesis proposed a robust centroid moment technique for hand gesture data to reduce k-dimensional space to achieve significant data reduction while retaining hand gesture information. SJM reduces k dimensional data from hand gesture skeleton data to three principal components (x, y, and zaxis). These components represent hand gesture moments. Researchers have proposed different methods of data reduction. One of the methods is PCA. PCA technique has similar accuracy compared to SJM. However, when new data is inserted, PCA must decompose the large datasets into a matrix of eigenvector and eigenvalue to describe their magnitude. Evaluation results using k-NN show that SJM has better accuracy than PCA for skeleton data. PCA has a higher uncertainty of mean error of 0.75 compared to SJM at only 0.01. In terms of accuracy, SJM shows 96% of prediction accuracy, similar to PCA using hand skeleton joints but with O(n) complexity compared to PCA with O(min(p3, n3)) where n is the data points in the dataset, and p is the features. Secondly, the importance and originality of this study are that it explores the complexity of hand gesture classification for incremental learning using a low-end device. Thus, this thesis presented a Convex Hull k-NN approach to optimize hand gesture classification complexity. The advantage of traditional k-NN is that it does not require preprocessing for a new batch of data. Many researchers have utilized k-NN to classify hand gestures in the past decades before moving to SVM and ANN. However, it is not practical for big data where the complexity is O(n). The solution is to extend the Convex Hull method into k-NN. The k-value is the smallest intersected region of hand gesture classes. The evaluation result of the t-test shows that P < 0.05 where there is a significant difference between Convex Hull SJM and Convex Hull PCA. Thus, the SJM is feasible for Convex Hull SJM k-NN and has the complexity of classification of O(c ∗ log(c)) for none intersected regions which is better than traditional k-NN O(n) and ANN O(nt ∗ (ij + jk + kl)) where i, j, k, and l are nodes, with t training examples and n epochs, and SVM O(n3). The experiment shows that SJM CH k-NN optimized computational complexity by O(c + i ∗ log(c + i)) for an incremental dataset in a real-time environment with high accuracy of 98% where c is the convex hull points. Finally, the third aim of this study is to investigate the uncertainty of hand gesture classification. The primary concern of Convex Hull SJM k-NN is the intersected region of the convex hull. Therefore in this thesis, density mapping for the intersected convex hulls is introduced in the CNS system. Using Convex Hull SJM k-NN with density mapping reduces 76% of data. The t-test result shows that SJM data and SJM Density Mapping data have a significant difference of P < 0.05. The F1 score results of this experiment show that Convex Hull SJM k-NN with Density Mapping has 94% of accuracy. The results indicate that Density Mapping reduces the data size into a fixed data frame for intersected convex hulls.

Text FSKTM 2022 2 UPMIR.pdf Download (1MB)

Actions (login required)

View Item