Two-Phase Defect Detection Using Clustering and Classification Methods

Ha Manh Tran, Tuan Anh Nguyen, Son Thanh Le, Giang Vu Truong Huynh, Tuan Bao Lam

Abstract


Autonomous fault management of network and distributed systems is a challenging research problem and attracts many research activities. Solving this problem heavily depends on expertise knowledge and supporting tools for monitoring and detecting defects automatically. Recent research activities have focused on machine learning techniques that scrutinize system output data for mining abnormal events and detecting defects. This paper proposes a two-phase defect detection for network and distributed systems using log messages clustering and classification. The approach takes advantage of K-means clustering method to obtain abnormal messages and random forest method to detect the relationship of the abnormal messages and the existing defects. Several experiments have evaluated the performance of this approach using the log message data of Hadoop Distributed File System (HDFS) and the bug report data of Bug Tracking System (BTS). Evaluation results have disclosed some remarks with lessons learned.


Full Text:

PDF


DOI: http://dx.doi.org/10.21553/rev-jec.296

Copyright (c) 2022 REV Journal on Electronics and Communications


ISSN: 1859-378X

Copyright © 2011-2024
Radio and Electronics Association of Vietnam
All rights reserved