Paper Info
Title
Mining top-k frequent items in a data stream with flexible sliding windows
Abstract
We study the problem of finding the k most frequent items in a stream of items for the recently proposed max-frequency measure. Based on the properties of an item, the max-frequency of an item is counted over a sliding window of which the length changes dynamically. Besides being parameterless, this way of measuring the support of items was shown to have the advantage of a faster detection of bursts in a stream, especially if the set of items is heterogeneous. The algorithm that was proposed for maintaining all frequent items, however, scales poorly when the number of items becomes large. Therefore, in this paper we propose, instead of reporting all frequent items, to only mine the top-k most frequent ones. First we prove that in order to solve this problem exactly, we still need a prohibitive amount of memory (at least linear in the number of items). Yet, under some reasonable conditions, we show both theoretically and empirically that a memory-efficient algorithm exists. A prototype of this algorithm is implemented and we present its performance w.r.t. memory-efficiency on real-life data and in controlled experiments with synthetic data.
Year
DOI
Venue
2010
10.1145/1835804.1835842
KDD
Keywords
Field
DocType
top-k frequent item,controlled experiment,max-frequency measure,real-life data,performance w,frequent item,prohibitive amount,faster detection,memory-efficient algorithm,length changes dynamically,synthetic data,data stream,data stream mining,sliding window
Data mining,Data stream mining,Sliding window protocol,Computer science,Data stream,Synthetic data,Artificial intelligence,Machine learning
Conference
Citations 
PageRank 
References 
20
0.72
8
Authors
2
Name
Order
Citations
PageRank
Hoang Thanh Lam11088.49
Toon Calders2133393.66