Name
Abstract | ||
|---|---|---|
Recurrent Neural Networks (RNN) are widely used for various prediction tasks on sequences such as text, speed signals, program traces, and system logs. Due to RNNs' inherently sequential behavior, one key challenge for the effective adoption of RNNs is to reduce the time spent on RNN inference and to increase the scope of a prediction. This work introduces CFG-guided compressed learning, an approach that creatively integrates Context-Free Grammar (CFG) and online tokenization into RNN learning and inference for streaming inputs. Through a hierarchical compression algorithm, it compresses an input sequence to a CFG and makes predictions based on the compressed sequence. Its algorithm design employs a set of techniques to overcome the issues from the myopic nature of online tokenization, the tension between inference accuracy and compression rate, and other complexities. Experiments on 16 real-world sequences of various types validate that the proposed compressed learning can successfully recognize and leverage repetitive patterns in input sequences, and effectively translate them into dramatic (1-1762x) inference speedups as well as much (1-7830x) expanded prediction scope, while keeping the inference accuracy satisfactory. |
Year | DOI | Venue |
|---|---|---|
2021 | 10.1109/ICDM51629.2021.00125 | 2021 21ST IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM 2021) |
Keywords | DocType | ISSN |
recurrent neural networks, data compression, context free grammar, tokenization | Conference | 1550-4786 |
Citations | PageRank | References |
0 | 0.34 | 0 |
Authors | ||
6 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Hui Guan | 1 | 0 | 0.34 |
| Umana Chaudhary | 2 | 0 | 0.34 |
| Yuanchao Xu | 3 | 6 | 3.77 |
| Lin Ning | 4 | 5 | 2.11 |
| Lijun Zhang | 5 | 0 | 0.34 |
| Xipeng Shen | 6 | 2025 | 118.55 |