Show simple item record

dc.contributor.authorAstekin, M.
dc.contributor.authorZengin, H.
dc.contributor.authorSözer, Hasan
dc.description.abstractAnomaly detection is a valuable feature for detecting and diagnosing faults in large-scale, distributed systems. These systems usually provide tens of millions of lines of logs that can be exploited for this purpose. However, centralized implementations of traditional machine learning algorithms fall short to analyze this data in a scalable manner. One way to address this challenge is to employ distributed systems to analyze the immense amount of logs generated by other distributed systems. We conducted a case study to evaluate two unsupervised machine learning algorithms for this purpose on a benchmark dataset. In particular, we evaluated distributed implementations of PCA and K-means algorithms. We compared the accuracy and performance of these algorithms both with respect to each other and with respect to their centralized implementations. Results showed that the distributed versions can achieve the same accuracy and provide a performance improvement by orders of magnitude when compared to their centralized versions. The performance of PCA turns out to be better than K-means, although we observed that the difference between the two tends to decrease as the degree of parallelism increases.en_US
dc.description.sponsorshipCloud Computing and Big Data Laboratory (B3LAB) of TUBITAK-BILGEM ; Software Research Laboratory (SRL) of Ozyegin University
dc.relation.ispartof2018 IEEE International Conference on Big Data (Big Data)
dc.titleEvaluation of distributed machine learning algorithms for anomaly detection from large-scale system logs: a case studyen_US
dc.typeConference paperen_US
dc.contributor.departmentÖzyeğin University
dc.contributor.authorID(ORCID 0000-0002-2968-4763 & YÖK ID 23178) Sözer, Hasan
dc.contributor.ozuauthorSözer, Hasan
dc.subject.keywordsLog analysisen_US
dc.subject.keywordsDistributed systemsen_US
dc.subject.keywordsParallel processingen_US
dc.subject.keywordsAnomaly detectionen_US
dc.subject.keywordsBig dataen_US
dc.subject.keywordsMachine learningen_US
dc.relation.publicationcategoryConference Paper - International - Institutional Academic Staff

Files in this item


There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record

Share this page