Name
Abstract | ||
|---|---|---|
This paper revisits set containment join (SCJ) problem, which uses the subset relationship (i.e., $$\subseteq$$) as condition to join set-valued attributes of two relations and has many fundamental applications in commercial and scientific fields. Existing in-memory algorithms for SCJ are either signature-based or prefix-tree-based. The former incurs high CPU cost because of the enumeration of signatures, while the latter incurs high space cost because of the storage of prefix trees. This paper proposes a new adaptive parameter-free in-memory algorithm, named as frequency-hashjoin or $${\mathsf {FreshJoin}}$$ in short, to evaluate SCJ efficiently. $${\mathsf {FreshJoin}}$$ builds a flat index on-the-fly to record three kinds of signatures (i.e., two least frequent elements and a hash signature whose length is determined adaptively by the frequencies of elements in the universe set). The index consists of two sparse inverted indices and two arrays which record hash signatures of all sets in each relation. The index is well organized such that $${\mathsf {FreshJoin}}$$ can avoid enumerating hash signatures. The rationality of this design is explained. And, the time and space cost of the proposed algorithm, which provide a rule to choose $${\mathsf {FreshJoin}}$$ from existing algorithms, are analyzed. Experiments on 16 real-life datasets show that $${\mathsf {FreshJoin}}$$ usually reduces more than 50% of space cost while remains as competitive as the state-of-the-art algorithms in running time. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1007/s41019-019-00107-y | Data Science and Engineering |
Keywords | Field | DocType |
Set containment join, Frequency hash, Join algorithm | Discrete mathematics,Data mining,Computer science,Spacetime,Enumeration,Prefix,Hash function,Universe,Adaptive algorithm | Journal |
Volume | Issue | ISSN |
4 | 4 | 2364-1185 |
Citations | PageRank | References |
1 | 0.35 | 0 |
Authors | ||
7 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| jizhou luo | 1 | 98 | 9.86 |
| Wei Zhang | 2 | 1 | 0.35 |
| Shengfei Shi | 3 | 82 | 15.87 |
| Hong Gao | 4 | 1086 | 120.07 |
| Jianzhong Li | 5 | 63 | 24.23 |
| Wei Wu | 6 | 4 | 3.13 |
| Shouxu Jiang | 7 | 116 | 14.83 |